You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "chungen0126 (via GitHub)" <gi...@apache.org> on 2026/08/18 20:57:16 UTC
[PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
chungen0126 opened a new pull request, #11053:
URL: https://github.com/apache/ozone/pull/11053
## What changes were proposed in this pull request?
This is the OEP doc for S3 WORM (Object Lock) feature.
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15945
## How was this patch tested?
No tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3955707684
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
Review Comment:
@fmorg-git , would you mind sharing some knowledge with @chungen0126 about how to expose new permissions for Ranger, and what are the major tasks needed for Ranger integration, including UI changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-5786270793
## Future: FSO buckets and Object Lock
We may need to support **FILE_SYSTEM_OPTIMIZED (FSO)** buckets eventually. The current OEP correctly scopes v1 to **OBS**, but it would help to add a short **“Future: FSO”** note so readers do not assume OBS handlers generalize for free.
**FSO is not natively compatible with the same WORM story as flat OBS keys.** AWS Object Lock is S3-object/version oriented; FSO adds a **file/directory split**, **path-based identity**, **rename**, **recursive/batch delete**, and **background purge** (`DirectoryDeletingService`, `OMDirectoriesPurgeRequestWithFSO`, etc.). Several of those paths have **no AWS S3 analogue**, so product and compliance rules need explicit decisions before implementation.
### Design decisions to make
- **Scope of lock:** Files only (`FileTable` / `OmKeyInfo`) vs directory entries (`DirectoryTable` / `OmDirectoryInfo`).
- **Clients:** S3G only vs also **OFS/Ozone FS** (rename/delete/overwrite bypass S3 action mapping unless OM enforces WORM on every RPC).
- **Rename / move:** Block when locked vs allow and **carry** lock metadata on the same `objectID`; rename-over-existing-key vs Put overwrite rules.
- **Tree delete:** Prefix delete when a descendant is locked; align partial failure with `DeleteObjects` semantics.
- **Legacy FSO** buckets vs current FSO layout before allowing `objectLockEnabled`.
### Engineering pointers (when FSO is in scope)
1. **Duplicate handler surface:** WORM must be wired into **`*WithFSO`** create/commit/delete/MPU paths and **`OMFileCreateRequestWithFSO`**, not only OBS `OMKey*Request`. A shared **`WormPolicyEvaluator`** (or similar) reduces risk of one missed handler becoming a bypass.
2. **Background jobs are high risk:** Directory/key **purge and pending-deletion** services must respect lock state; async purge must not delete locked files under “deleted” directory markers.
3. **Rename:** `OMKeyRenameRequestWithFSO` / batch rename — treat as first-class WORM surface (DELETE+CREATE ACL-wise; compliance impact if rename is allowed without policy).
4. **Enforcement authority:** **OM `validateAndUpdateCache`** (and equivalent on FSO handlers) should remain authoritative; Ranger governs lock **administration** and governance **bypass**, not replacement for metadata checks on OFS clients.
5. **Testing:** FSO-specific integration — tree delete with one locked file, rename, MPU on paths, purge service, optional o3fs if in scope.
Suggest a brief **Non-goal today / Future work** subsection in the OEP pointing here so OBS v1 can ship without implying FSO is a small delta.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3954644467
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
Review Comment:
`CLEAR_LEGAL_HOLD` can be covered by `PUT_LEGAL_HOLD`, since AWS S3 doesn't have CLEAR_LEGAL_HOLD permission too, so let's avoid introduce a new extra permission.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3987541297
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
Review Comment:
Object locking can be configured and managed via both the Ozone API and the S3 API.
- Ozone API: It uses OzoneBucket#setRetentionConfig. If no default retention is set, the rule field within retention will simply be null.
- S3 API: It is managed through PutBucketObjectLockConfiguration, which basically follows the standard S3 specifications. You can refer to the official [AWS S3 documentation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLockConfiguration.html) for more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4290107658
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
> @fmorg-git , how about we always carry the S3 action in RequestContext during Ranger authentication? Whether these action will be used in Ranger will depends on ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition is true or not. The cost of this solution is we need to extract and set the S3 action in S3g, and pass to OM for each request, and if it's an expensive operation, then it may have performance impact.
@ChenSammi - I think that's a good idea to enforce actions even if STS is not enabled. I created a Jira here: https://issues.apache.org/jira/browse/HDDS-17054. It seems weird to require STS flag for Object Lock to work.
@chungen0126 @ChenSammi - I'm not sure I follow the discussion on native acls - wouldn't they work in similar way to Ranger, such that the actions are checked first, then the access types? Otherwise, READ_ACL could perhaps overgrant a number of S3 api's.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4356657966
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
Review Comment:
Can we rename this to RetentionDuration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4357768649
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
Review Comment:
I believe the API name is GetObjectLockConfiguration, the S3 permission name is GetBucketObjectLockConfiguration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3841281466
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
Review Comment:
Is it really true for Ozone. In AWS S3, it is being made sure to be immutable at all levels or atleast audited or blocked including admin/operators users, but in Ozone, this may not be true, currently design talks about a happy path using `validateAndUpdateCache` when hit on normal S3/ozone APIs. In Ozone there are other ways where someone can directly edit rocksDb or delete a key meta for a locked key. Another way, where `ozone repair `command also allows to restore OM's DB directory from a ratis snapshot taken before the lock was applied.
Probably the design should talk about behavior when such actions can happen in Ozone , that what is trusted boundary and upto what extent the data is protected because otherwise this design may not suffice the complete compliance audit implementation.
**Can think of below:**
- Whether `ozone repair` subcommands should refuse to operate on Raft indexes that carry lock state changes without an explicit unsafe flag.
- Whether locked keys should be detected on OM startup via cross-check between RocksDB state and a tamper-evident external audit log — so that a manual DB-copy recovery that silently removes a lock can at least be detected on the next start.
- an operator can run `skip-ratis-transaction` at the Raft index of a SetRetention(compliance) entry across all 3 OMs while stopped, then restart, and the audit trail of the lock is gone (though RocksDB may still hold applied state
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
Review Comment:
In contrast to AWS S3, how do we present this to compliance auditors? Or should there be a "one-way" flag that, once set, cannot be cleared?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
Review Comment:
Is it possible to move the OM leader's clock forward: The WORM check in `validateAndUpdateCache` will compare `retainUntilDate` (stored) against "now". "Now" is whatever the OM leader believes the time is.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4036146281
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
Review Comment:
> In a distributed development environment, it is possible for your application to initiate several updates on the same object at the same time. Your application might initiate several multipart uploads using the same object key. For each of these uploads, your application can then upload parts and send a complete upload request to Amazon S3 to create the object. When the buckets have S3 Versioning enabled, completing a multipart upload always creates a new version. When you initiate multiple multipart uploads that use the same object key in a versioning-enabled bucket, the current version of the object is determined by which upload started most recently (createdDate).
AWS S3 explicitly allows multiple concurrent multipart uploads for the same key.
If we only reject requests at CreateMultipartUpload, an MPU initiated before the lock is applied could still be completed after the lock becomes active. This would overwrite the protected object and break the WORM guarantee. Therefore, we still need to validate the WORM state during CompleteMultipartUpload to ensure strict immutability.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3954617490
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
Review Comment:
Bucket level retention configuration is different than object level. It support days and years, not the retainUntilDate.
https://docs.aws.amazon.com/cli/latest/reference/s3api/put-object-lock-configuration.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3987416006
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
Review Comment:
Updated
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+
+### Impact on Write and Delete Flows
+#### Impact on Put Object Flow
+
+Put Flow:
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+#### Impact on Multipart Upload Flow
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. During Initiate Multipart Upload and Put Part phases, WORM checks are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. During the Complete Multipart Upload phase, final validation occurs in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+#### Impact on Delete Object Flow
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
Review Comment:
Bucket & Volume Removal:
1. In Ozone, a bucket cannot be deleted if it is non-empty (still contains keys/directories), and similarly, a volume cannot be deleted if it contains buckets.
2. As noted in Non-Goals, FSO buckets are excluded from this design:
- Target use case: S3 Object Lock is an S3-centric compliance feature targeted at OBS buckets.
- Directory operations: FSO deletes directory trees by moving them to the deletedDirectoryTable in $O(1)$ time without validating deep children. Checking WORM locks recursively would break this $O(1)$ guarantee and require expensive tree traversals.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
Review Comment:
> In a distributed development environment, it is possible for your application to initiate several updates on the same object at the same time. Your application might initiate several multipart uploads using the same object key.
AWS S3 explicitly allows multiple concurrent multipart uploads for the same key.
If we only reject requests at CreateMultipartUpload, an MPU initiated before the lock is applied could still be completed after the lock becomes active.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "shuan1026 (via GitHub)" <gi...@apache.org>.
shuan1026 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4064112619
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,557 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required RetentionMode retentionMode = 1;
+ required TimeUnit timeUnit = 2;
+ required uint64 duration = 3;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionDate & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly specify the `x-amz-bypass-governance-retention: true` header in the delete or overwrite request.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**:
+> _Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them due to unsupporting Versioning, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+Preliminary assessments indicate that the additional metadata lookups impose an acceptable impact on overall system throughput and latency. Metrics will be continuously monitored and tuned as necessary.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions.
+
+By leveraging the **STS action-matching framework**, Ozone maps standard S3 API actions (e.g., `BypassGovernanceRetention`, `PutObjectLegalHold`) directly to standard Ranger primitive access types (e.g., `WRITE` and `READ`). This fine-grained, action-level authorization enforces the Principle of Least Privilege **without requiring custom dedicated access types**. It effectively prevents unauthorized tampering or removal of locks, ensuring robust compliance and data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+* **System Clock Tampering**: Artificially manipulating or rolling back the system clock (NTP/OS time) on OM or DataNode hosts to prematurely expire object retention periods or evade time-based WORM validation.
+* **Ranger Super Admin Privilege Abuse**: Modifying, disabling, or overriding Ozone authorization policies via Apache Ranger admin credentials (or direct access to Ranger’s backend database), allowing privileged operators to alter bucket WORM flags or grant unrestricted deletion permissions outside normal enforcement workflows.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+* **Table Schema Changes**: Since all newly introduced fields in the Bucket Table (`objectLockEnabled`, `defaultRetention`) and Key Table (`retentionConfig`, `legalHold`) are defined as `optional` in Protobuf, they maintain natural forward and backward compatibility across rolling upgrades.
+* **OM Version Leveling**: OM version leveling is introduced primarily because this feature adds new OM APIs (such as `setRetentionConfig` and `addLegalHold`). The new OM version allows clients to determine whether the target OM cluster supports executing these Object Lock APIs during rolling upgrades.
+
+## Alternatives
+
+### 1. Why Not Use Native ACLs for Object Lock Management?
+
+* **Proposal**:
+ Support Native ACLs alongside or instead of Ranger to configure and manage Object Lock states (e.g., using Ozone native ACLs or POSIX-like ACLs at the Volume/Bucket/Key level).
+
+* **Why Rejected**:
+ 1. **Conflicting Security & Authorization Philosophies**:
+ Native ACLs in Ozone are primarily designed for granular, resource-centric permissions (often granting permissions directly to specific objects/buckets or inherited via path hierarchies). In contrast, Object Lock is an enterprise compliance feature that requires strict, role-based access governance (RBAC/ABAC). Compliance controls like WORM and Legal Holds are inherently tied to compliance officers, security auditors, or legal teams—not individual data owners.
+ 2. **Risk of Decentralized & Inconsistent Governance**:
+ Native ACLs allow object creators or bucket owners to adjust permissions. If lock controls were exposed through Native ACLs, standard bucket administrators or key owners could modify or override lock-related ACLs locally. Centralizing lock authorization exclusively in **Apache Ranger** ensures that compliance policies are managed from a unified pane of glass, preventing local policy drift or accidental misconfigurations.
+ 3. **Operational Complexity**:
+ Supporting dual authorization models (Ranger + Native ACLs) for critical compliance features introduces unnecessary complexity, potential security loopholes, and ambiguous resolution hierarchies during audits. Excluding Native ACLs ensures clean boundary enforcement.
+
+## Plan
+
+The implementation of S3 Object Lock in Apache Ozone is divided into five structured phases covering the 11 core functional requirements. This ensures incremental delivery, clean service boundaries, and testability at each stage.
+
+### Phase 1: Metadata Infrastructure & Schema Definitions
+
+*Goal: Establish protobuf schemas, in-memory domain models, and RocksDB table serialization codecs.*
+
+1. **Protobuf Definitions (`OmClientProtocol.proto`)**:
+ - Introduce retention domain messages:
+ - `RetentionMode` enum (`GOVERNANCE = 1`, `COMPLIANCE = 2`).
+ - `Rule` message (`retentionMode`, `timeUnit`, `duration`).
+ - `EventHold` message (`enabled`, `rule`).
+ - `RetentionConfig` message (`rule`, `eventHold`).
+ - Extend `BucketInfo` (OBS buckets only):
+ - `required bool objectLockEnabled = 24 [default = false];`
+ - `optional RetentionConfig defaultRetention = 25;`
+ - Extend `KeyInfo` & `KeyInfoProtoLight`:
+ - `optional RetentionConfig retentionConfig = 23;`
+ - `optional bool legalHold = 24 [default = false];`
+
+2. **Domain Models & Helpers (`hadoop-ozone/common`)**:
+ - Update `OmBucketInfo` / `OmBucketInfo.Builder` with getters, setters, and protobuf translation.
+ - Update `OmKeyInfo` / `OmKeyInfo.Builder` with getters, setters, and protobuf translation.
+ - Introduce new class `Retention` to encapsulate bucket rules of retention:
+ - `fromProto` to convert from protobuf to java class.
+ - `toProto` to convert from java class to protobuf.
+ - Introduce `RetentionUtil` to provide utility methods for retention calculations:
+ - Calculate `RetainUntilDate` based on `RetentionConfig` and current system time.
+ - Validate retention rules (e.g., duration > 0, valid time unit, etc.).
+
+---
+
+### Phase 2: Ranger Integration & Action Mapping
+
+*Goal: Enable fine-grained access control separating standard data mutating permissions from compliance actions.*
+
+1. **Ranger Action Mapping & UI Validation (`ozone.json`)**:
+ - Do **NOT** create new underlying Ranger access types. Retain primitive access types (`READ`, `WRITE`, `DELETE`) and leverage the STS action-matching framework (`action-matches`).
+ - Register the 7 S3 Object Lock actions in `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json`:
+ - `GetBucketObjectLockConfiguration` (Bucket level: requires `READ`)
+ - `PutBucketObjectLockConfiguration` (Bucket level: requires `WRITE`)
+ - `GetObjectRetention` (Key level: requires `READ`)
+ - `PutObjectRetention` (Key level: requires `WRITE`)
+ - `GetObjectLegalHold` (Key level: requires `READ`)
+ - `PutObjectLegalHold` (Key level: requires `WRITE`)
+ - `BypassGovernanceRetention` (Key level: requires `WRITE` / `DELETE`)
+ - Enable design-time validation in Ranger React UI to ensure administrators cannot grant compliance actions without selecting their required primitive permissions.
+
+2. **Payload-Aware String Matcher Condition Evaluator**:
+ - Register a dedicated custom condition evaluator (e.g., `RangerOzoneValueMatcher`, extending `RangerAbstractConditionEvaluator`) in `ranger-servicedef-ozone.json` under `policyConditions`.
+ - Update `RangerOzoneAuthorizer` and request context handling to extract sensitive payload attributes and inject them into `RangerAccessRequest.getContext()`:
+ - `s3:object-lock-legal-hold`: Extracted from the request body (`ON` vs. `OFF`).
+ - `s3:object-lock-mode`: Extracted from the request body (`GOVERNANCE` vs. `COMPLIANCE`).
+ - `s3:x-amz-bypass-governance-retention`: Extracted from request headers (`true` vs. `false`).
+ - Implement runtime string matching within the evaluator to enforce value-level role segregation:
+ - Allow decoupling privileged operations (e.g., granting `PutObjectLegalHold` exclusively when `s3:object-lock-legal-hold == "ON"`, while restricting `OFF` to compliance officers).
+ - Prevent accidental or unauthorized lock lock-in by enforcing `s3:object-lock-mode == "GOVERNANCE"` for standard operators, denying `COMPLIANCE` configuration.
+
+3. **S3 Gateway & IAM Action Resolution**:
+ - Update `S3GAction` in `hadoop-ozone/s3gateway`: add audit actions for all lock configuration operations.
+ - Update `S3GActionIamMapper` in `hadoop-ozone/s3gateway`: map the new audit actions to standard IAM S3 actions (`GetBucketObjectLockConfiguration`, `PutBucketObjectLockConfiguration`, `GetObjectRetention`, `PutObjectRetention`, `GetObjectLegalHold`, `PutObjectLegalHold`, `BypassGovernanceRetention`).
+ - Update `IamSessionPolicyResolver.S3Action` in `hadoop-ozone/common`: register the actions with their resource scopes (Bucket vs. Object) and base permissions (`READ`, `WRITE`).
+ - Propagate `s3Action` via `S3Auth` through OM RPC to `RequestContext.s3Action` for evaluation by `RangerOzoneAuthorizer`.
+
+---
+
+### Phase 3: Object Lock Management APIs
+
+*Goal: Implement S3 REST endpoints, OM client RPC protocols, and OM HA consensus handlers for configuring and querying locks.*
+
+#### 1. Bucket Object Lock Configuration
+- **`PutBucketObjectLockConfiguration`**:
+ - *S3G*: Route `PUT /{bucket}?object-lock`. Parse `ObjectLockConfiguration` XML (`ObjectLockEnabled`, optional `DefaultRetention`).
+ - *RPC*: Add `SetBucketObjectLockConfigRequest` / `Response` in `OmClientProtocol.proto`.
+ - *OM Handler (`OMBucketSetObjectLockConfigRequest`)*:
+ - `preExecute`: Validate Ranger action `PutBucketObjectLockConfiguration`. Fail-fast on invalid parameters.
+ - `validateAndUpdateCache`: Under bucket write lock, check bucket exists and layout is OBS. Ensure immutability: if `objectLockEnabled` is already `true`, it cannot be toggled to `false`. Set `objectLockEnabled` and update `defaultRetention`. Commit to `BucketTable`.
+- **`GetBucketObjectLockConfiguration`**:
+ - *S3G*: Route `GET /{bucket}?object-lock`. Return `ObjectLockConfiguration` XML.
+ - *OM / Metadata Reader*: Authorize `GetBucketObjectLockConfiguration`. Read from `BucketTable`. Return `404 ObjectLockConfigurationNotFoundError` if Object Lock is not enabled.
+
+#### 2. Object Retention Management
+- **`PutObjectRetention`**:
+ - *S3G*: Route `PUT /{bucket}/{key}?retention`. Parse `Retention` XML (`Mode`, `RetainUntilDate`). Parse header `x-amz-bypass-governance-retention`.
+ - *RPC*: Add `SetObjectRetentionRequest` / `Response` in `OmClientProtocol.proto`.
+ - *OM Handler (`OMKeySetRetentionRequest`)*:
+ - `preExecute`: Authorize `PutObjectRetention`. If bypass header is set, authorize `BypassGovernanceRetention`.
+ - `validateAndUpdateCache`: Under key write lock, verify bucket has `objectLockEnabled == true`.
+ - **Compliance Mode**: If existing key is in COMPLIANCE, the new `RetainUntilDate` MUST be >= existing `RetainUntilDate` (cannot shorten duration or switch mode).
+ - **Governance Mode**: If shortening retention or removing, caller must have `BypassGovernanceRetention` permission and send the bypass flag.
+ - Update `OmKeyInfo.retentionConfig` in `KeyTable`.
+- **`GetObjectRetention`**:
+ - *S3G*: Route `GET /{bucket}/{key}?retention`. Return `Retention` XML.
+ - *OM / Metadata Reader*: Authorize `GetObjectRetention`. Read from `KeyTable`. Return `404 NoSuchObjectLockConfiguration` if no retention policy is applied.
+
+#### 3. Object Legal Hold Management
+- **`PutObjectLegalHold`**:
+ - *S3G*: Route `PUT /{bucket}/{key}?legal-hold`. Parse `LegalHold` XML (`Status`: `ON` / `OFF`).
+ - *RPC*: Add `SetObjectLegalHoldRequest` / `Response` in `OmClientProtocol.proto`.
+ - *OM Handler (`OMKeySetLegalHoldRequest`)*:
+ - `preExecute`: Authorize `PutObjectLegalHold`.
+ - `validateAndUpdateCache`: Under key write lock, verify bucket has `objectLockEnabled == true`. Update `OmKeyInfo.legalHold` (`true` for `ON`, `false` for `OFF`) in `KeyTable`.
+- **`GetObjectLegalHold`**:
+ - *S3G*: Route `GET /{bucket}/{key}?legal-hold`. Return `LegalHold` XML (`Status`: `ON` / `OFF`).
+ - *OM / Metadata Reader*: Authorize `GetObjectLegalHold`. Read from `KeyTable`. Return `404 NoSuchObjectLockConfiguration` if unconfigured.
+
+---
+
+### Phase 4: WORM Enforcement on Data Operations
+
+*Goal: Enforce dual-gate WORM protection on all data-mutating operations to guarantee immutability on single-version OBS objects.*
+
+#### 1. Check Lock on Put / Copy Operations
+- **Bucket Default Retention Inheritance**:
+ - When creating a *new* key in an Object-Lock-enabled bucket:
+ - If bucket has `defaultRetention`, automatically attach `RetentionConfig` to the newly committed `OmKeyInfo`.
+ - If request provides explicit retention/legal hold headers (`x-amz-object-lock-*`), validate permissions and apply them on creation.
+- **Overwrite Protection (Two-Phase Validation)**:
+ - **`OMKeyCreateRequest.preExecute`** (Fail-fast):
+ - Check if key already exists in `KeyTable`.
+ - If exists and locked (`legalHold == true` OR unexpired `retentionConfig` without valid Governance bypass): immediately fail with `WORMProtectionException` (maps to S3 `403 AccessDenied`). Avoids unnecessary Raft consensus.
+ - **`OMKeyCommitRequest.validateAndUpdateCache`** (Linearizability):
+ - Under bucket/key write lock, perform authoritative WORM check against committed `KeyTable`.
+ - Reject commit if target is actively locked. Keys that fail to commit remain in `OpenKeyTable` and are reclaimed by background open-key cleanup.
+
+#### 2. Check Lock on Delete Operations
+- **Single Delete (`DeleteObject`)**:
+ - In `OMKeyDeleteRequest.validateAndUpdateCache` under key write lock:
+ - Lookup target key in `KeyTable`. If absent, return standard idempotent success (`204 No Content`).
+ - If key exists:
+ - Active Legal Hold (`legalHold == true`) -> Reject with `WORMProtectionException`.
+ - Active Retention:
+ - `COMPLIANCE` -> Reject unconditionally.
+ - `GOVERNANCE` -> Allow only if caller has `BypassGovernanceRetention` permission AND header `x-amz-bypass-governance-retention: true` is present; otherwise reject.
+ - If unlocked or bypass authorized, proceed with deletion and quota reclaim.
+- **Batch Delete (`DeleteObjects`)**:
+ - In `S3BatchDeleteRequest` / `MultiDeleteEndpoint`: evaluate WORM status per key.
Review Comment:
These two classes don't exist in master. The path is BucketEndpoint.multiDelete -> OzoneBucket.deleteKeys -> OMKeysDeleteRequest. Checking in S3G also misses KeyLifecycleService, which submits DeleteKeysRequest directly into Ratis and never goes through S3G. Could the check go in OMKeysDeleteRequest.validateAndUpdateCache?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4278205874
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
Or to just simply have the requirement that both the Ozone STS flag and the Ranger flag are set in order to use Object Lock? I think there is already the requirement that Ranger flag is set.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4257260808
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
Review Comment:
Make sense.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4349812257
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
Correct, OM's administrator check does not depend on the authorizer, so "Ranger has no administrator role" was not the right reason. I'd still keep Compliance under Ranger policies instead of requiring an Ozone administrator:
1. A Compliance retention set by mistake is not stuck forever: `ozone repair om object-lock` can remove or downgrade it offline.
2. `ozone.administrators` is cluster wide, so anyone who manages Compliance for a single bucket would have to be an administrator of the whole cluster.
3. It would remove Ranger's fine-grained delegation, for example granting `PutObjectRetention` with `action-matches` to a compliance team on specific buckets only, which is the main reason to use Ranger for this feature.
The risk you describe (action matcher off by default, so any `WRITE` can set Compliance) is called out in the Ranger "Important" note and the Trusted Boundary section, and the Phase 2 user documentation will cover it. The fix is to enable the action matcher and use `action-matches` on `WRITE` items.
Updated the wording of that line to give these reasons instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4357230734
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
Review Comment:
There is one limitation about bucket locking default configuration,
`If you specify both a default retention period (Days or Years) and a DefaultEventHold, the event hold duration can't exceed the default retention period.
`
https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-s3-bucket-defaultretention.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4358137535
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Setting Compliance only needs `PutObjectRetention` (`WRITE`), as in AWS S3, so it can be delegated per bucket with Ranger policies.
+ Requiring an Ozone administrator, as with native ACLs, would make it cluster wide.
+ Only `action-matches` can limit who sets it. No Ranger permission can remove a Compliance retention; only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+##### Access Type Mapping
+
+Policy items without `action-matches` are checked by access type only.
+
+| S3 Action | Resource | Access type | Description |
+|:-----------------------------------|:---------|:---------------------|:----------------------------------------------------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` | Read the bucket's Object Lock settings. |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE` | Set the bucket's default retention (mode, period, event hold). |
+| `GetObjectRetention` | Key | `READ` | Read the retention mode, date and event hold. |
+| `PutObjectRetention` | Key | `WRITE` | Set, extend, shorten or remove a retention; set or release an event hold. |
+| `GetObjectLegalHold` | Key | `READ` | Read the legal hold. |
+| `PutObjectLegalHold` | Key | `WRITE` | Set the legal hold `ON` or `OFF`. |
+| `BypassGovernanceRetention` | Key | `WRITE` and `DELETE` | Bypass Governance retention. |
+
+##### `action-matches`
+
+* With `enableActionMatcherInPoliciesCondition`, a policy item can have an `action-matches` condition: a list of S3 actions (`*` and `Prefix*` allowed)
+ that `RangerActionMatcher` compares with the action of the request.
+* It only narrows an item: the item applies when both the access type and the action match. Items without it behave as before.
+* Example: grant a legal team `WRITE` with `action-matches: PutObjectLegalHold`, so it can change legal holds but not retention.
+* S3 Gateway sets the action in `S3GActionIamMapper`, and OM passes it to Ranger in the `RequestContext`. OM only adds the dependent actions below.
+* Today S3 Gateway only sends the action when `ozone.s3g.sts.http.enabled=true`. This design depends on [HDDS-17054](https://issues.apache.org/jira/browse/HDDS-17054), which sends it for every S3 request.
+* For requests without an S3 action (ofs, o3fs, Ozone CLI, Java API), see [Trusted Boundary](#trusted-boundary-and-compliance-scope).
+
+##### Dependent Permissions
+
+As in AWS S3 ([required permissions for S3 API operations](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html)),
+some requests also need a second action, depending on their headers or on the key:
+
+| S3 API | Main S3 action | Header or condition | Dependent S3 action | Access type | If denied |
+|:----------------------------------------------------------|:---------------------|:-------------------------------------------------------------------------------------------------------|:-------------------------------------------|:------------------------|:--------------------------------------------|
+| `CreateBucket` | `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | `PutBucketObjectLockConfiguration` | `WRITE` (bucket) | Request fails |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `PutObject` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | `PutObjectRetention` | `WRITE` (key) | Request fails |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `PutObject` | `x-amz-object-lock-legal-hold` | `PutObjectLegalHold` | `WRITE` (key) | Request fails |
+| `GetObject`, `HeadObject` | `GetObject` | The key has a retention or a legal hold | `GetObjectRetention`, `GetObjectLegalHold` | `READ` (key) | OM drops the lock information, so S3 Gateway omits the `x-amz-object-lock-*` headers; the read succeeds |
+| `DeleteObject`, `DeleteObjects` | `DeleteObject` | `x-amz-bypass-governance-retention: true` on a key under Governance retention | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails (`DeleteObjects`: per-key `AccessDenied`) |
+| `PutObjectRetention` | `PutObjectRetention` | Shortens or removes a Governance retention | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails |
+| `PutObject`, `CopyObject`, `CompleteMultipartUpload`, rename (native clients) | `PutObject` | Overwrites or renames a key under Governance retention with `bypassGovernanceRetention` | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails |
+
+The dependent actions use the same access types as data (`READ` / `WRITE`, and `DELETE` for the bypass), so they mostly matter for policy items with `action-matches`.
+With such items, a data writer cannot set its own retention with a header, and a retention manager cannot upload data.
+
+**How it works**: S3 Gateway sends one RPC with the main action. OM calls `checkAccess` once per action, each with one access type and one action, and every call must pass.
Review Comment:
Implementation wise, let's think about how can we get the check in one call for both PutObject and PutObjectRetention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3955407846
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
Review Comment:
Reject upload in CreateMultipartUpload is good enough for the MPU, CompleteMultipartUpload and UploadPart can be untouched.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3955407846
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
Review Comment:
I think reject upload in CreateMultipartUpload is good enough for the MPU, CompleteMultipartUpload and UploadPart can be untouched.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3963595743
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
Review Comment:
Also, like @ChenSammi says, I believe you need to separate the authorization from the value being set. For example, the same `PutObjectLegalHold` authorization can be used with values `ON` or `OFF` in the API (see https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLegalHold.html)
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
Review Comment:
I believe `BypassGovernanceRetention` should be checked at the key level.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4290659618
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
Currently, we have no immediate requirement for WORM support.
I agree with @ChenSammi proposal of using READ and owner and the admin rather than using READ_ACL and WRITE_ACL, but I will leave it up to the community to decide. Our cluster ACL model is different so even if we decided to backport this in the future, we will need to adapt it to our ACL model.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279070853
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
Review Comment:
Updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266773854
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
Review Comment:
If we restrict the authorization to Ranger only, then the root or administrator role is not applied, there is no root or ozone administrator in Ranger.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4358006836
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Setting Compliance only needs `PutObjectRetention` (`WRITE`), as in AWS S3, so it can be delegated per bucket with Ranger policies.
+ Requiring an Ozone administrator, as with native ACLs, would make it cluster wide.
Review Comment:
"Requiring an Ozone administrator, as with native ACLs, would make it cluster wide" I'm not very clear about this statement, could you do a little further explanation?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4022161741
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+
+### Impact on Write and Delete Flows
+#### Impact on Put Object Flow
+
+Put Flow:
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+#### Impact on Multipart Upload Flow
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. During Initiate Multipart Upload and Put Part phases, WORM checks are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. During the Complete Multipart Upload phase, final validation occurs in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+#### Impact on Delete Object Flow
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
Review Comment:
This can be divided into two parts:
1. **Bucket or volume removal:**
Currently, a bucket cannot be deleted if it still contains keys. Therefore, this is not impacted.
2. **Directory removal:**
The current design does not support FSO or legacy buckets, so it will not touch upon directory removal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4031808438
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,559 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required RetentionMode retentionMode = 1;
+ required TimeUnit timeUnit = 2;
+ required uint64 duration = 3;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+* **retentionDate**: Represents the expiration timestamp of the retention lock. While clients or API calls specify retention duration in terms of days or years, Ozone calculates the definitive expiration date upon applying the rule and persists it as a timestamp into key table.
+* **legalHold**: Indicates whether an explicit legal hold is active on the key.
+
+Two new fields: retentionDate & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional string retentionDate = 23;
+ optional RetentionConfig retentionConfig = 24;
+ optional bool legalHold = 25 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+| :--- | :--- | :--- | :--- |
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` / `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
Review Comment:
quick clarification - does it mean `BypassGovernanceRetention` requires `WRITE` and `DELETE` together, similar to how `PutObject` requires `WRITE` and `CREATE` together? If so, may want to say `and` instead of `/`
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,559 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required RetentionMode retentionMode = 1;
+ required TimeUnit timeUnit = 2;
+ required uint64 duration = 3;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+* **retentionDate**: Represents the expiration timestamp of the retention lock. While clients or API calls specify retention duration in terms of days or years, Ozone calculates the definitive expiration date upon applying the rule and persists it as a timestamp into key table.
+* **legalHold**: Indicates whether an explicit legal hold is active on the key.
+
+Two new fields: retentionDate & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional string retentionDate = 23;
+ optional RetentionConfig retentionConfig = 24;
+ optional bool legalHold = 25 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+| :--- | :--- | :--- | :--- |
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` / `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly specify the `x-amz-bypass-governance-retention: true` header in the delete or overwrite request.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**:
+> _Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them due to unsupporting Versioning, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+Preliminary assessments indicate that the additional metadata lookups impose an acceptable impact on overall system throughput and latency. Metrics will be continuously monitored and tuned as necessary.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions. Dedicated Access Types (BYPASS_GOVERNANCE, PUT_LEGAL_HOLD, etc.) enforce the Principle of Least Privilege, preventing unauthorized tampering or removal of locks and enhancing data immutability.
Review Comment:
I think there won't be dedicated access types anymore right?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,559 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required RetentionMode retentionMode = 1;
+ required TimeUnit timeUnit = 2;
+ required uint64 duration = 3;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+* **retentionDate**: Represents the expiration timestamp of the retention lock. While clients or API calls specify retention duration in terms of days or years, Ozone calculates the definitive expiration date upon applying the rule and persists it as a timestamp into key table.
+* **legalHold**: Indicates whether an explicit legal hold is active on the key.
+
+Two new fields: retentionDate & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional string retentionDate = 23;
+ optional RetentionConfig retentionConfig = 24;
+ optional bool legalHold = 25 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
Review Comment:
nit - I believe Put requires `CREATE` and `WRITE` access.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,559 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required RetentionMode retentionMode = 1;
+ required TimeUnit timeUnit = 2;
+ required uint64 duration = 3;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+* **retentionDate**: Represents the expiration timestamp of the retention lock. While clients or API calls specify retention duration in terms of days or years, Ozone calculates the definitive expiration date upon applying the rule and persists it as a timestamp into key table.
+* **legalHold**: Indicates whether an explicit legal hold is active on the key.
+
+Two new fields: retentionDate & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional string retentionDate = 23;
+ optional RetentionConfig retentionConfig = 24;
+ optional bool legalHold = 25 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+| :--- | :--- | :--- | :--- |
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` / `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
Review Comment:
curious, what is the reason for this additional complexity? AWS doesn't work like this. FYI @ChenSammi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4392075088
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
Review Comment:
Hi @ChenSammi , Before changing this I want to check the intent, because the key owner is not part of authorization today.
OmKeyInfo.ownerName is stored, but OzoneNativeAuthorizer only skips ACL checks for the volume and bucket owner (OMClientRequest.checkAcls only puts those two into the context). The key owner has no role. key-level access is controlled by key ACLs.
So adding the key owner here means Object Lock would be the first place where the key owner gets implicit rights, and only for lock requests. Two consequences:
My view: for WORM data I would rather keep the bucket owner as the gate. The bucket owner decides who may change locks in that bucket, and that is a clearer control than "whoever wrote the key". I am open to the key owner if you have a use case in mind; in that case I would treat it as a separate change to the native authorizer rather than part of the default Object Lock model.
What do you think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub) via issues" <is...@ozone.apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4449254297
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,637 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account, so it uses the owner or the Ozone administrator role instead.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; every change to a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+* Releasing a Compliance retention: in AWS S3 no one can shorten or remove it. Ozone lets an Ozone administrator do it through `PutObjectRetention`, in both models
+ (see [Releasing a Compliance Retention](#4-releasing-a-compliance-retention)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can overwrite or delete the key before it expires. Only an Ozone administrator can shorten it, remove it or change it to Governance.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (Compliance: only an Ozone administrator).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Lock Settings by Resource
+
+| Setting | Bucket default (`PutObjectLockConfiguration`) | Key (`PutObjectRetention`, `PutObjectLegalHold`, `x-amz-object-lock-*` headers) |
+|:-----------|:------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
+| Mode | `GOVERNANCE` or `COMPLIANCE` | `GOVERNANCE` or `COMPLIANCE` |
+| Fixed date | A duration in days or years; a new key gets creation time + duration | `RetainUntilDate` |
+| Event hold | `DefaultEventHold` has only a duration, no status; a new key gets the hold `ON` with that duration | `EventHold` `ON` / `OFF` with `EventHoldDuration` |
+| Legal hold | Not available | `ON` / `OFF` |
+
+A bucket default has no `RetainUntilDate` and no legal hold. As in AWS S3 ([DefaultRetention](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-s3-bucket-defaultretention.html)),
+a default with both a duration and `DefaultEventHold` is only valid when the event hold duration does not exceed the default duration (`400 InvalidRequest` otherwise).
+A default with only `DefaultEventHold` is valid.
+
+How the settings combine on a key:
+* A key is locked while `legalHold = ON`, while its event hold is `ON`, or while its effective `RetainUntilDate` is in the future. Legal hold and retention are independent, so all three must be gone before the key is unlocked.
+* `x-amz-object-lock-*` headers at creation replace the bucket default entirely; without them, the key gets the bucket default (mode, `retentionDate` = creation time + duration, and the event hold).
+* Effective `RetainUntilDate`:
+ * Fixed retention: `retentionDate`
+ * Event hold `ON`: `max(retentionDate, now + EventHoldDuration)`, so it moves forward over time
+ * Release without `RetainUntilDate`: `retentionDate = max(retentionDate, release time + EventHoldDuration)`
+ * Release with `RetainUntilDate`: `retentionDate = requested RetainUntilDate` (earlier than the effective date only with the [release permission](#event-hold-trigger-workflow))
+ * Shorter `EventHoldDuration` while `ON`: `retentionDate = max(retentionDate, now + old duration)` first, then the new duration applies
+
+The table, the combination rules and the formulas were verified against AWS S3 (us-east-1, 2026-10-28) in Governance mode.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `duration` and `defaultEventHold`, a new key gets `retentionDate` = creation time + `duration` and an active event hold.
+ With only `defaultEventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Making it earlier needs the **release permission**:
+the bypass flag and the bypass permission for `GOVERNANCE`, and an Ozone administrator for `COMPLIANCE` (see [Releasing a Compliance Retention](#4-releasing-a-compliance-retention)).
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`
+ (`InvalidRequest`; `InvalidArgument` for a `RetainUntilDate` before now + duration; `MalformedXML` for a retention without a mode).
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the release permission (`403` without it).
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the release permission, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the release permission.
+ * With the release permission, `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the release permission, otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the release permission, otherwise `403`. While the hold is `ON`, it gets `400` even with the release permission: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+#### Retention Updates
+
+`PutObjectRetention` on a key that already has a retention follows AWS S3 (see [Event Hold Trigger Workflow](#event-hold-trigger-workflow)).
+"Longer" and "shorter" compare the requested `RetainUntilDate` with the current effective date:
+
+| Change | `GOVERNANCE` | `COMPLIANCE` |
+|:------------------------------------------------------------------------------------|:-------------------------------------------------|:-------------------------------------|
+| Longer retention (later `RetainUntilDate`, longer event hold duration, `EventHold=ON`) | Allowed | Allowed |
+| Shorter event hold duration while the hold is `ON` | Allowed; the effective date is kept | Allowed; the effective date is kept |
+| Shorter retention (earlier `RetainUntilDate`, also on release) or removal (an empty retention) | Needs the bypass; while the hold is `ON`, an earlier `RetainUntilDate` must come with `EventHold` | Ozone administrator only; the same `EventHold` rule applies |
+| Release the event hold (`EventHold=OFF` without `RetainUntilDate`) | Allowed | Allowed |
+| Change the mode | `GOVERNANCE` → `COMPLIANCE`: allowed | `COMPLIANCE` → `GOVERNANCE`: Ozone administrator only |
+
+Who may send `PutObjectRetention` depends on the model: see [Native ACLs](#1-native-acls) and [Ranger](#2-ranger).
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional RetentionDuration duration = 2;
+ optional RetentionDuration defaultEventHold = 3; // only a duration: new keys start with the hold ON
+ }
+
+ message RetentionDuration {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+
+message EventHold {
+ required bool enabled = 1;
Review Comment:
should EventHold.enabled have a default, similar to how KeyInfo.eventHold has a default?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,637 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account, so it uses the owner or the Ozone administrator role instead.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; every change to a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+* Releasing a Compliance retention: in AWS S3 no one can shorten or remove it. Ozone lets an Ozone administrator do it through `PutObjectRetention`, in both models
+ (see [Releasing a Compliance Retention](#4-releasing-a-compliance-retention)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can overwrite or delete the key before it expires. Only an Ozone administrator can shorten it, remove it or change it to Governance.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (Compliance: only an Ozone administrator).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Lock Settings by Resource
+
+| Setting | Bucket default (`PutObjectLockConfiguration`) | Key (`PutObjectRetention`, `PutObjectLegalHold`, `x-amz-object-lock-*` headers) |
+|:-----------|:------------------------------------------------------------------------|:---------------------------------------------------------------------------------|
+| Mode | `GOVERNANCE` or `COMPLIANCE` | `GOVERNANCE` or `COMPLIANCE` |
+| Fixed date | A duration in days or years; a new key gets creation time + duration | `RetainUntilDate` |
+| Event hold | `DefaultEventHold` has only a duration, no status; a new key gets the hold `ON` with that duration | `EventHold` `ON` / `OFF` with `EventHoldDuration` |
+| Legal hold | Not available | `ON` / `OFF` |
+
+A bucket default has no `RetainUntilDate` and no legal hold. As in AWS S3 ([DefaultRetention](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-s3-bucket-defaultretention.html)),
+a default with both a duration and `DefaultEventHold` is only valid when the event hold duration does not exceed the default duration (`400 InvalidRequest` otherwise).
+A default with only `DefaultEventHold` is valid.
+
+How the settings combine on a key:
+* A key is locked while `legalHold = ON`, while its event hold is `ON`, or while its effective `RetainUntilDate` is in the future. Legal hold and retention are independent, so all three must be gone before the key is unlocked.
+* `x-amz-object-lock-*` headers at creation replace the bucket default entirely; without them, the key gets the bucket default (mode, `retentionDate` = creation time + duration, and the event hold).
+* Effective `RetainUntilDate`:
+ * Fixed retention: `retentionDate`
+ * Event hold `ON`: `max(retentionDate, now + EventHoldDuration)`, so it moves forward over time
+ * Release without `RetainUntilDate`: `retentionDate = max(retentionDate, release time + EventHoldDuration)`
+ * Release with `RetainUntilDate`: `retentionDate = requested RetainUntilDate` (earlier than the effective date only with the [release permission](#event-hold-trigger-workflow))
+ * Shorter `EventHoldDuration` while `ON`: `retentionDate = max(retentionDate, now + old duration)` first, then the new duration applies
+
+The table, the combination rules and the formulas were verified against AWS S3 (us-east-1, 2026-10-28) in Governance mode.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `duration` and `defaultEventHold`, a new key gets `retentionDate` = creation time + `duration` and an active event hold.
+ With only `defaultEventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Making it earlier needs the **release permission**:
+the bypass flag and the bypass permission for `GOVERNANCE`, and an Ozone administrator for `COMPLIANCE` (see [Releasing a Compliance Retention](#4-releasing-a-compliance-retention)).
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`
+ (`InvalidRequest`; `InvalidArgument` for a `RetainUntilDate` before now + duration; `MalformedXML` for a retention without a mode).
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the release permission (`403` without it).
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the release permission, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the release permission.
+ * With the release permission, `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the release permission, otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the release permission, otherwise `403`. While the hold is `ON`, it gets `400` even with the release permission: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+#### Retention Updates
+
+`PutObjectRetention` on a key that already has a retention follows AWS S3 (see [Event Hold Trigger Workflow](#event-hold-trigger-workflow)).
+"Longer" and "shorter" compare the requested `RetainUntilDate` with the current effective date:
+
+| Change | `GOVERNANCE` | `COMPLIANCE` |
+|:------------------------------------------------------------------------------------|:-------------------------------------------------|:-------------------------------------|
+| Longer retention (later `RetainUntilDate`, longer event hold duration, `EventHold=ON`) | Allowed | Allowed |
+| Shorter event hold duration while the hold is `ON` | Allowed; the effective date is kept | Allowed; the effective date is kept |
+| Shorter retention (earlier `RetainUntilDate`, also on release) or removal (an empty retention) | Needs the bypass; while the hold is `ON`, an earlier `RetainUntilDate` must come with `EventHold` | Ozone administrator only; the same `EventHold` rule applies |
+| Release the event hold (`EventHold=OFF` without `RetainUntilDate`) | Allowed | Allowed |
+| Change the mode | `GOVERNANCE` → `COMPLIANCE`: allowed | `COMPLIANCE` → `GOVERNANCE`: Ozone administrator only |
+
+Who may send `PutObjectRetention` depends on the model: see [Native ACLs](#1-native-acls) and [Ranger](#2-ranger).
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional RetentionDuration duration = 2;
+ optional RetentionDuration defaultEventHold = 3; // only a duration: new keys start with the hold ON
+ }
+
+ message RetentionDuration {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+
+message EventHold {
+ required bool enabled = 1;
+ required RetentionDuration duration = 2;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and every change to a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, releasing a Compliance retention needs an Ozone administrator (see [Releasing a Compliance Retention](#4-releasing-a-compliance-retention)),
+and keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| `COMPLIANCE`: setting it on a key, `GOVERNANCE` → `COMPLIANCE`, any `PutObjectRetention` on a `COMPLIANCE` key (extending it, shortening, removing or downgrading it, or releasing its event hold), or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger to delegate Compliance per bucket.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every `PutObjectRetention` on a Compliance key needs an administrator: setting it, extending it (a later date or a longer event hold duration), releasing its event hold,
+ and shortening, removing or downgrading it. One rule for Compliance is easier to apply than exceptions for an event hold release with or without a `RetainUntilDate`.
+ Without it, an owner could lock a key for up to 100 years that only an administrator can unlock. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+ There is no separate "remove lock" operation: shortening or removing a Governance retention is a `PutObjectRetention` that also needs `BypassGovernanceRetention`
+ (see [Dependent Permissions](#dependent-permissions)), so a principal with `WRITE` alone can set or extend locks but not shorten or remove a Governance retention.
+* Setting Compliance only needs `PutObjectRetention` (`WRITE`), as in AWS S3, and only `action-matches` can limit who sets it, so a Ranger policy can delegate it to a team for specific buckets.
+ Native ACLs need an Ozone administrator instead, and `ozone.administrators` is one list for the whole cluster, so it cannot be limited to a bucket.
+ Releasing a Compliance retention still needs an Ozone administrator: OM checks `ozone.administrators` after the Ranger check,
+ and no Ranger permission alone is enough (see [Releasing a Compliance Retention](#4-releasing-a-compliance-retention)).
+
+##### Access Type Mapping
+
+Policy items without `action-matches` are checked by access type only.
+The S3 actions are the AWS permission names; `GetBucketObjectLockConfiguration` / `PutBucketObjectLockConfiguration` authorize the `GetObjectLockConfiguration` / `PutObjectLockConfiguration` APIs.
+
+| S3 Action | Resource | Access type | Description |
+|:-----------------------------------|:---------|:---------------------|:----------------------------------------------------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` | Read the bucket's Object Lock settings. |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE` | Set the bucket's default retention (mode, duration, event hold). |
+| `GetObjectRetention` | Key | `READ` | Read the retention mode, date and event hold. |
+| `PutObjectRetention` | Key | `WRITE` | Set, extend, shorten or remove a retention; set or release an event hold. |
+| `GetObjectLegalHold` | Key | `READ` | Read the legal hold. |
+| `PutObjectLegalHold` | Key | `WRITE` | Set the legal hold `ON` or `OFF`. |
+| `BypassGovernanceRetention` | Key | `WRITE` and `DELETE` | Bypass Governance retention. |
+
+##### `action-matches`
+
+* With `enableActionMatcherInPoliciesCondition`, a policy item can have an `action-matches` condition: a list of S3 actions (`*` and `Prefix*` allowed)
+ that `RangerActionMatcher` compares with the action of the request.
+* It only narrows an item: the item applies when both the access type and the action match. Items without it behave as before.
+* Example: grant a legal team `WRITE` with `action-matches: PutObjectLegalHold`, so it can change legal holds but not retention.
+* S3 Gateway sets the action in `S3GActionIamMapper`, and OM passes it to Ranger in the `RequestContext`. OM only adds the dependent actions below.
+* Today S3 Gateway only sends the action when `ozone.s3g.sts.http.enabled=true`. This design depends on [HDDS-17054](https://issues.apache.org/jira/browse/HDDS-17054), which sends it for every S3 request.
+* Requests without an S3 action (ofs, o3fs, Ozone CLI, Java API) never match an item with a specific `action-matches` value.
+ They are allowed only by items without the condition (or with `*`); otherwise they are denied (see [Trusted Boundary](#trusted-boundary-and-compliance-scope)).
+
+##### Dependent Permissions
+
+As in AWS S3 ([required permissions for S3 API operations](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html)),
+some requests also need a second action, depending on their headers, the bypass flag or the bucket:
+
+| S3 API | Main S3 action | Header or condition | Dependent S3 action | Access type | If denied |
+|:----------------------------------------------------------|:---------------------|:-------------------------------------------------------------------------------------------------------|:-------------------------------------------|:------------------------|:--------------------------------------------|
+| `CreateBucket` | `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | `PutBucketObjectLockConfiguration` | `WRITE` (bucket) | Request fails |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `PutObject` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | `PutObjectRetention` | `WRITE` (key) | Request fails |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `PutObject` | `x-amz-object-lock-legal-hold` | `PutObjectLegalHold` | `WRITE` (key) | Request fails |
+| `GetObject`, `HeadObject` | `GetObject` | The bucket has Object Lock enabled | `GetObjectRetention`, `GetObjectLegalHold` | `READ` (key) | OM drops the lock information, so S3 Gateway omits the `x-amz-object-lock-*` headers; the read succeeds |
+| `DeleteObject`, `DeleteObjects` | `DeleteObject` | `x-amz-bypass-governance-retention: true` | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails (`DeleteObjects`: per-key `AccessDenied`) |
+| `PutObjectRetention` | `PutObjectRetention` | `x-amz-bypass-governance-retention: true` (required to shorten or remove a Governance retention) | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails |
+| `PutObject`, `CopyObject`, `CompleteMultipartUpload`, rename (native clients) | `PutObject` | `bypassGovernanceRetention` (required to overwrite or rename a key under Governance retention) | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails |
+
+The dependent actions use the same access types as the direct lock changes (`READ` or `WRITE`; `WRITE` and `DELETE` for the bypass), so they mostly matter for policy items with `action-matches`.
+With such items, a data writer cannot set its own retention with a header, and a retention manager cannot upload data.
+
+**How it works**: S3 Gateway sends one RPC with the main S3 action. In `preExecute`, the OM request runs the normal ACL check for that action, then looks at the request itself:
+each Object Lock header or flag it carries adds one (access type, S3 action) check on the same resource, for example (`WRITE`, `PutObjectRetention`) for retention headers,
+or (`WRITE`, `BypassGovernanceRetention`) and (`DELETE`, `BypassGovernanceRetention`) for the bypass flag. OM calls `checkAccess` once per check.
+The main action always comes from S3 Gateway; OM never maps an API to its action. OM only adds dependent actions, as the existing `DEPENDENT_S3_ACTIONS` map already does,
+because their conditions are headers and flags that OM parses anyway.
+
+Whether a check is needed depends only on what the request carries (a header or the bypass flag), not on whether the key is currently locked, so `preExecute` does not read the key.
+The WORM check in `validateAndUpdateCache` decides whether the lock allows the operation.
+`GetObject` / `HeadObject` also check `GetObjectRetention` and `GetObjectLegalHold` with `READ`, before the lookup, when the bucket has Object Lock enabled.
+
+What happens when a check is denied is in the "If denied" column of the table above. The dependent checks apply to the target resource only (the parent volume check is not repeated),
+and they also run for requests without an S3 action, by access type alone. Listings do not return lock information, so they need no check per key.
+
+Each `checkAccess` is an in-process evaluation against the Ranger plugin's cached policies and one Ranger audit record, not a round trip.
+This generalizes the existing `DEPENDENT_S3_ACTIONS` map in `OmMetadataReader` (today `GetObjectAttributes` → `GetObject`): `OMClientRequest.checkAcls` takes the checks declared by the request,
+and `OmMetadataReader.checkAcls` accepts the S3 action per call instead of taking it only from the thread-local `S3Auth`.
Review Comment:
> `OMClientRequest.checkAcls` takes the checks declared by the request,
and `OmMetadataReader.checkAcls` accepts the S3 action per call instead of taking it only from the thread-local `S3Auth`.
just curious how this will be done?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4398813282
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
Review Comment:
Added "Lock Settings by Resource" after Terminology: a bucket default vs key table (bucket has no legal hold and uses a duration).
I verified the table and every formula against AWS S3 today .
Also tightened the proto: the bucket default stores only a duration (defaultEventHold), since AWS DefaultEventHold has no ON/OFF status.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4398979009
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Setting Compliance only needs `PutObjectRetention` (`WRITE`), as in AWS S3, so it can be delegated per bucket with Ranger policies.
+ Requiring an Ozone administrator, as with native ACLs, would make it cluster wide.
Review Comment:
ozone.administrators is one list for the whole cluster, so it cannot be limited to a bucket. With Ranger a policy can let a team set Compliance on specific buckets only, with native ACLs the only choice is the cluster-wide administrator list. Added into design doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4036081830
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,559 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required RetentionMode retentionMode = 1;
+ required TimeUnit timeUnit = 2;
+ required uint64 duration = 3;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+* **retentionDate**: Represents the expiration timestamp of the retention lock. While clients or API calls specify retention duration in terms of days or years, Ozone calculates the definitive expiration date upon applying the rule and persists it as a timestamp into key table.
+* **legalHold**: Indicates whether an explicit legal hold is active on the key.
+
+Two new fields: retentionDate & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional string retentionDate = 23;
+ optional RetentionConfig retentionConfig = 24;
+ optional bool legalHold = 25 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+| :--- | :--- | :--- | :--- |
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` / `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
Review Comment:
Thanks for the feedback! Actually, AWS IAM does support this kind of value-level condition matching for S3.
According to the [AWS IAM Condition Operators documentation](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_condition_operators.html), S3 provides condition keys that allow restricting specific actions based on request values.
The reason I included this additional complexity is based on use cases like the one described in this [AWS re:Post article: Denying removal of Amazon S3 Object Lock Legal Hold](https://repost.aws/articles/ARHP6wZxPyQ-6xGawE5MFPuw/denying-removal-of-amazon-s3-object-lock-legal-hold). The article demonstrates how administrators separate the permissions for turning a legal hold `ON` versus `OFF` by leveraging the `StringEquals` condition operator. As mentioned in the quote:
> As legal hold has no expiration date, users may wish to use this mode to apply an indefinite lock on objects they wish to protect from accidental or malicious deletion. In this scenario, it may be desirable to restrict permissions to remove legal hold from objects. You can do this with a condition key in the Condition element of an IAM policy, specifically "StringEquals": "s3:object-lock-legal-hold": "OFF" applied to the action "PutObjectLegalHold".
That being said, I am completely open to discussing this. If we feel that `PutObjectLegalHold` (without distinguishing between ON/OFF payloads) is enough for Ozone at this stage, we can definitely drop this custom condition evaluator to keep the initial implementation simple.
Let me know what @fmorg-git and @ChenSammi think!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4019553488
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,470 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ optional RetentionMode retentionMode = 1;
+ optional uint64 days = 2;
+ optional uint64 years = 3;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted standard Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Action Authorization**: Following the Ranger action-matching model introduced via the [STS](ozone-sts.md), Object Lock actions map directly to AWS S3 action names (without the `s3:` prefix). Access control separates underlying Ranger permissions (e.g., `READ`, `WRITE`) from fine-grained compliance actions:
+ * `GetBucketObjectLockConfiguration`
+ * `PutBucketObjectLockConfiguration`
+ * `GetObjectRetention`
+ * `PutObjectRetention`
+ * `GetObjectLegalHold`
+ * `PutObjectLegalHold`
+ * `BypassGovernanceRetention`
+
+#### 1. Legal Hold Access Control
+
+Authorization is decoupled from the payload value being set, aligning strictly with AWS S3 specifications:
+
+* **`GetObjectLegalHold`**: Allows querying the current Legal Hold status of an object.
+* **`PutObjectLegalHold`**: Governs both setting and clearing Legal Hold status (status ON or OFF). Users must be granted the `PutObjectLegalHold` action (along with `WRITE` permission) in Ranger to toggle the hold.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BypassGovernanceRetention`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Evaluated at the **Key level** via the **`BypassGovernanceRetention`** action.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BypassGovernanceRetention` entitlement in Ranger for that key. Without this permission, even administrators using standard client tools cannot modify or remove the locked object.
+
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3987578470
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **Future Evolution (With Versioning)**: When S3 Versioning is supported, Object Lock and Versioning will be tightly coupled following AWS S3 semantics: once Object Lock is enabled, Versioning cannot be suspended or disabled.
+> * **Bucket Creation Semantics**:
Review Comment:
updated
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **Future Evolution (With Versioning)**: When S3 Versioning is supported, Object Lock and Versioning will be tightly coupled following AWS S3 semantics: once Object Lock is enabled, Versioning cannot be suspended or disabled.
+> * **Bucket Creation Semantics**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+Preliminary assessments indicate that the additional metadata lookups impose an acceptable impact on overall system throughput and latency. Metrics will be continuously monitored and tuned as necessary.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions. Dedicated Access Types (BYPASS_GOVERNANCE, PUT_LEGAL_HOLD, etc.) enforce the Principle of Least Privilege, preventing unauthorized tampering or removal of locks and enhancing data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+Because this design alters the schemas of Bucket Table and Key Table, OM (Ozone Manager) version leveling will be introduced to maintain forward and backward compatibility across rolling upgrades.
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-5965033664
Hi @ChenSammi, @fmorg-git , @devmadhuu , @sumitagrawl , @jojochuang would you like to take another look? If there are no further comments or questions, I’d like to start moving forward with the implementation. Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4357999035
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
Review Comment:
I feel we need both write and delete permission for object lock change too, as object lock change includes remove the lock, which has the equal effect as bypass.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4356578818
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
Review Comment:
This conflicts with above "setting or changing a Compliance retention needs an Ozone administrator " statement. We need to pick either ozone administrator is allowed or an offline operation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4357778921
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
Review Comment:
PutBucketObjectLockConfiguration -> PutObjectLockConfiguration
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4267052021
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
Review Comment:
use uint64 for retentionDate to avoid time convert every time when it's accessed.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
Review Comment:
use uint64 for retentionDate to avoid time convert every time when it's accessed and compared.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266764067
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
Review Comment:
OBS is used a few places in this doc, some time it's Object Storage, some time it's OBS bucket type.
Let's use OBS only for bucket type, and use "Object Storage", or instead "Ozone" here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266918042
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
Review Comment:
It's not a good idea to support a `ozone sh volume delete -r` command, to minic the AWS account close situation. Whether and how Ozone should support an account close situation, is out of the scope of this feature.
For this feature, the real problem is if Compliance Mode is put on a object, is there anyone can remove it?
If we want to follow the AWS rule, then instead of provide the volume deletion command, provide a offline Compliance Mode update command per key or batch is more appropriate. @chungen0126 , there is this "ozone repair" command category, where commands are provided to fix the problems offline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4278331972
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
Review Comment:
Done, renamed to KeyRetention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4321053613
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,589 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets, with authorization through Ozone native ACLs or Apache Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in Ozone is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design supports both Ozone native ACLs and Apache Ranger to manage Object Lock permissions on OBS buckets.
+Native ACLs are used by deployments without Ranger and allow the core logic to be implemented and tested independently of the Ranger integration.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, with permission and access control through native ACLs or Ranger.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Value-Level Ranger Authorization: a payload-aware Ranger condition evaluator (for example, allowing a legal hold to be set `ON` but not `OFF`) is deferred to a future phase.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Authorization: Ozone has no root account.
+ * With native ACLs, changing Object Lock settings requires `WRITE_ACL` and bypassing Governance retention is limited to Ozone administrators (see [Native ACL Mapping](#1-native-acl-mapping)).
+ * With Ranger, AWS-style action-level authorization for S3 requests requires both `ozone.s3g.sts.http.enabled` and `enableActionMatcherInPoliciesCondition` to be enabled; both are `false` by default (see [Prerequisite for S3 Fine-Grained Authorization](#3-prerequisite-for-s3-fine-grained-authorization)).
+ * Otherwise, and for native clients, Ranger falls back to `READ_ACL` / `WRITE_ACL`, and bypassing Governance retention requires `WRITE_ACL`, `WRITE` and `DELETE` (see [Fallback: Base Permission Mode](#4-fallback-base-permission-mode)).
+ * A Compliance retention cannot be removed through any API; it can only be changed offline with `ozone repair` (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until a principal allowed to change the legal hold (`WRITE_ACL` on the key, plus the `PutObjectLegalHold` action in Ranger Fine-Grained Mode) explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+* **Usage**:
+ * Used to preserve data for litigation, investigations or audits, when it is not known in advance how long the data must be kept.
+ * A legal hold has no mode and no expiration date. It is independent of retention: a key can have a legal hold, a retention period, or both,
+ and it stays locked until the legal hold is removed and the retention period has expired.
+ * A legal hold can be placed on a key in an Object Lock enabled bucket even if the key has no retention.
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * The bucket default can have both `rule` and `eventHold`. In that case a new key gets `retentionDate` = creation time + `rule` and an active event hold;
+ the key stays locked until the event hold is released and `retentionDate` has passed. With only `eventHold`, `retentionDate` is unset until the hold is released.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the latest of the existing `retentionDate`, the requested `RetainUntilDate`, and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **authorization (native ACLs or Ranger)** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Base Permissions**: Object Lock operations are checked against the existing `READ_ACL` / `WRITE_ACL` types for both native ACLs and Ranger, so no new ACL type is needed (see below).
+* **Fine-Grained S3 Action Matching (Ranger)**: For requests through S3 Gateway, Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Native ACL Mapping
+
+Changing Object Lock state is treated as a permission change, so it requires `WRITE_ACL` rather than `WRITE`.
+A user who can only write or delete data cannot remove or shorten a lock.
+
+| Operation | Resource Level | Required Native ACL |
+|:---------------------------------------|:---------------|:-----------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ_ACL` |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE_ACL` |
+| `GetObjectRetention` | Key | `READ_ACL` |
+| `PutObjectRetention` | Key | `WRITE_ACL` |
+| `GetObjectLegalHold` | Key | `READ_ACL` |
+| `PutObjectLegalHold` | Key | `WRITE_ACL` |
+| Bypass Governance retention | Key | Ozone administrator only |
+
+**Ozone administrator** means a user listed in `ozone.administrators` or a member of a group listed in `ozone.administrators.groups`
+(if neither is set, only the user that starts the OM is an administrator). This is the same check used for other admin-only OM operations.
+* The native authorizer already grants every ACL to Ozone administrators, so they also pass the `READ_ACL` / `WRITE_ACL` checks above.
+* Bypass is limited to administrators because the native ACL model has no permission dedicated to it: any ACL type would also be grantable by bucket or key owners,
+ who could then remove the Governance protection from their own data.
+* Read-only administrators (`ozone.readonly.administrators`) and S3 administrators (`ozone.s3.administrators`) cannot bypass Governance retention.
+* The administrator role is cluster wide, so it cannot be scoped to a single volume or bucket. Deployments that need per-bucket bypass entitlement should use Ranger.
+* The bypass does not apply to Compliance retention or a legal hold; a legal hold must be removed explicitly with `PutObjectLegalHold`, which an administrator can do because it only requires `WRITE_ACL`.
+
+---
+
+#### 2. Ranger Authorization Modes
+
+With Ranger, every Object Lock request is authorized in one of two modes.
+S3 actions are an S3 protocol concept, so only S3 Gateway sets them; OM does not add S3 actions to requests itself.
+In both modes OM always checks the base access type (`READ_ACL` / `WRITE_ACL`), and the OM WORM state validation is the same.
+
+| Request source | `ozone.s3g.sts.http.enabled` | `enableActionMatcherInPoliciesCondition` | Mode |
+|:--------------------------------|:-----------------------------|:-----------------------------------------|:------------------------------------------------------------------------|
+| S3 Gateway | `true` | `true` | [Fine-Grained Mode](#3-prerequisite-for-s3-fine-grained-authorization) |
+| S3 Gateway | `false` | `true` | [Base Permission Mode](#4-fallback-base-permission-mode); denied by policy items with an action condition (see below) |
+| S3 Gateway | `false` | `false` | [Base Permission Mode](#4-fallback-base-permission-mode) |
+| S3 Gateway | `true` | `false` | [Base Permission Mode](#4-fallback-base-permission-mode) (not recommended, see below) |
+| Native Java API / CLI | any | any | [Base Permission Mode](#4-fallback-base-permission-mode); with the Ranger flag enabled, denied by policy items with an action condition |
+
+* **Fine-Grained Mode**: Ranger checks the S3 action (for example `PutObjectLegalHold`) and its base access type in the [mapping below](#6-action-to-ranger-access-type-mapping).
+* **Base Permission Mode**: Ranger checks only the base access type.
+
+**Requests without an action when the Ranger flag is enabled**:
+
+When `enableActionMatcherInPoliciesCondition=true`, the result for a request without an S3 action (S3 Gateway with the STS flag disabled, or a native client) depends on the matching Ranger policy item:
+
+| Matching policy item | Request with an S3 action | Request without an S3 action |
+|:----------------------------------------------|:---------------------------------------------------|:-----------------------------------------------------|
+| Has an action condition (e.g. `PutObjectLegalHold`) | Granted only if the action is listed and the access type matches | **Denied**: the action condition cannot be satisfied |
+| Has no action condition | Granted if the access type matches | Granted if the access type matches (Base Permission Mode) |
+
+* If the principal is granted Object Lock access only through policy items with an action condition, every Object Lock request without an action is denied,
+ including the requests that the same principal could send through S3 Gateway with both flags enabled.
+* If the principal is granted the access type through a policy item without an action condition, the request falls back to Base Permission Mode.
+* When `enableActionMatcherInPoliciesCondition=false`, action conditions are not evaluated at all, so every request is in Base Permission Mode.
+
+---
+
+#### 3. Prerequisite for S3 Fine-Grained Authorization
+
+> _**Important for S3 users**_: AWS-style action-level authorization for Object Lock (separate permissions for `PutObjectRetention`,
+> `PutObjectLegalHold`, `PutBucketObjectLockConfiguration` and `BypassGovernanceRetention`) is only available when **both** flags below are enabled.
+> Both are `false` by default, so a cluster with default settings authorizes S3 Object Lock requests in [Base Permission Mode](#4-fallback-base-permission-mode).
+
+```properties
+# S3 Gateway and OM: extract the S3 action and forward it to the authorizer
+ozone.s3g.sts.http.enabled=true
+# Ranger Ozone service definition: evaluate the S3 action in policy conditions
+ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+```
+
+**Why both flags are required**:
+* `ozone.s3g.sts.http.enabled` decides whether the S3 action reaches Ranger at all. Today S3 Gateway sets `S3Authentication.s3Action` and OM copies it into the `RequestContext` only when this flag is `true`
+ (`EndpointBase.applyS3Action`, `OmMetadataReader.maybeAddToContextFromThreadLocal`). The action was originally only needed for STS, which is why it is tied to this flag.
+* `enableActionMatcherInPoliciesCondition` decides whether Ranger uses the action. When it is `false`, Ranger's `RangerActionMatcher` ignores the action and only checks the access type.
+* If only the Ranger flag is enabled, S3 Gateway sends no action. Requests are denied by policy items with an action condition, and fall back to Base Permission Mode only through policy items without one
+ (see [Requests without an action when the Ranger flag is enabled](#2-ranger-authorization-modes)). S3 users granted Object Lock permissions by action would therefore be denied, so this combination is not recommended either.
+* If only the STS flag is enabled, Ranger ignores the action, so requests also fall back to Base Permission Mode.
+ This combination is not recommended: STS session policies restrict temporary credentials by S3 action (see [AWS STS Design for Ozone S3](ozone-sts.md)),
+ so without the Ranger flag those restrictions would not be enforced. This is also why the STS flag is not enabled by default.
+
+**What S3 users get in Fine-Grained Mode**:
+* Object Lock permissions can be granted separately, as in AWS IAM. For example, a legal team can be granted `PutObjectLegalHold` without `PutObjectRetention`,
+ and `BypassGovernanceRetention` can be granted to a smaller group than the one that sets retention.
+* STS session policies (AssumeRole) can include or exclude Object Lock actions, and `IamSessionPolicyResolver` restricts the temporary credential accordingly.
+* Each action still requires its base access type, so a wrong action mapping in S3 Gateway cannot reduce the check below `READ_ACL` / `WRITE_ACL`.
+
+**Operator guidance**:
+* Clusters that expose Object Lock to S3 users with different roles (for example, data writers, compliance officers and administrators) should enable both flags before enabling Object Lock on any bucket.
+* The STS flag only affects S3 Gateway requests. Native clients never carry an action, so they are denied by policy items with an action condition and are granted only through policy items without one (see [below](#4-fallback-base-permission-mode)).
+* The user documentation of this feature will describe both flags and the difference between the two modes.
+
+---
+
+#### 4. Fallback: Base Permission Mode
+
+Base Permission Mode is used for every request without an S3 action:
+requests from the native Java API or CLI, and S3 Gateway requests when either flag in the [previous section](#3-prerequisite-for-s3-fine-grained-authorization) is disabled.
+OM does not set an S3 action for these requests, so that the S3 action mapping stays in `S3GActionIamMapper` only.
+When the Ranger flag is enabled, these requests are granted only through policy items without an action condition
+(see [Requests without an action when the Ranger flag is enabled](#2-ranger-authorization-modes)).
+
+**Checks**:
+
+| Operation | Resource Level | Required Ranger Access Type |
+|:---------------------------------------|:---------------|:----------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ_ACL` |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE_ACL` |
+| `GetObjectRetention` | Key | `READ_ACL` |
+| `PutObjectRetention` | Key | `WRITE_ACL` |
+| `GetObjectLegalHold` | Key | `READ_ACL` |
+| `PutObjectLegalHold` | Key | `WRITE_ACL` |
+| Bypass Governance retention | Key | `WRITE_ACL`, `WRITE` and `DELETE` |
+
+**Why `READ_ACL` / `WRITE_ACL` instead of `READ` / `WRITE`**:
+* Without an S3 action, a `WRITE` check could not tell `PutObject` from `PutObjectLegalHold`, so any user who can write a key could also remove its legal hold or shorten its retention.
+* Changing a lock is treated as a permission change, so it uses the same access type as changing an ACL. Users with only `READ`, `WRITE`, `CREATE` or `DELETE` on a key cannot read or change its Object Lock settings.
+* This matches the [Native ACL Mapping](#1-native-acl-mapping), so the same operation needs the same access type with native ACLs and with Ranger.
+
+**Limitations compared with Fine-Grained Mode**:
+* All Object Lock changes on a resource need the same `WRITE_ACL`, so they cannot be granted separately. A principal that can set a legal hold can also set retention on the same key, and can bypass Governance retention if it also has `WRITE` and `DELETE`.
+* Governance retention therefore only protects keys from principals without `WRITE_ACL`, `WRITE` and `DELETE`. Deployments that need a separate bypass entitlement should use Fine-Grained Mode.
+* `WRITE_ACL` is also the access type for changing ACLs, so granting Object Lock management also grants that.
+* Unlike native ACLs, where bypass is limited to Ozone administrators, Ranger has no Ozone administrator role, so bypass is granted through a policy with `WRITE_ACL`, `WRITE` and `DELETE`.
+* Compliance retention is not affected: it cannot be removed in either mode.
+
+**Native clients when both flags are enabled**:
+* Native clients carry no S3 action. A policy item with an action condition cannot be satisfied by a request without an action, so it does not grant the request.
+* A principal whose `WRITE_ACL` is limited to `PutObjectLegalHold` through S3 Gateway is therefore denied all Object Lock operations through a native client;
+ it cannot use a native client to get around the action-level restrictions.
+* To let a principal manage Object Lock through a native client, grant the access type through a policy item without an action condition.
+ That item applies to S3 Gateway requests as well, so the principal gets every Object Lock operation allowed by the access type, without action-level separation.
+
+---
+
+#### 5. Ranger & S3 Gateway Component Updates
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (`READ_ACL`, `WRITE_ACL`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings, mapped to the same `READ_ACL` / `WRITE_ACL` base access types.
+
+---
+
+#### 6. Action to Ranger Access Type Mapping
+
+In Fine-Grained Mode, following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ_ACL` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE_ACL` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ_ACL` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE_ACL` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ_ACL` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE_ACL` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE_ACL`, `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 7. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable through the Ozone API. **No role, administrator, native ACL or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires. The only way to change it early is the offline `ozone repair` command described below.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Bypass Entitlement**: With native ACLs, the principal must be an Ozone administrator (`ozone.administrators`).
+ With Ranger, the principal needs `WRITE_ACL`, `WRITE` and `DELETE` on the target key, and in Fine-Grained Mode also the **`BypassGovernanceRetention`** action.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted through the Ozone API by any admin, native ACL or Ranger policy until the retention period expires.
+> It can only be changed offline with `ozone repair` (see below).
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+* Bucket links: S3 buckets are often links (for example `/s3v/bucket1` → `/vol1/bucket1`). Unlike `OMBucketSetPropertyRequest`, which rejects property changes on links,
+ `PutBucketObjectLockConfiguration` follows the link and stores the configuration on the source bucket, and `GetBucketObjectLockConfiguration` reads it from there.
+ Authorization is checked on both the link and the source bucket, like other requests that resolve links. Key operations already resolve links, so WORM checks always use the source bucket's configuration.
+
+**Object-Level APIs:**
Review Comment:
Thanks for pointing that out.
Added a matrix for each authorization model, based on the AWS page you linked (see "Access Control"):
- Ranger ("Dependent Permissions"): `PutObject` / `CopyObject` / `CreateMultipartUpload` with retention headers also need `PutObjectRetention`, and with the legal hold header also need `PutObjectLegalHold`. The table also covers `CreateBucket` with Object Lock, the lock headers on `GetObject` / `HeadObject`, and the Governance bypass on delete, overwrite and rename. With `action-matches`, a data writer cannot set its own retention with a header, and a retention manager cannot upload data.
- Native ACLs ("Extra Checks for Object Lock Headers"): the same headers need the volume or bucket owner, or an Ozone administrator for Compliance, so a user who can only write data cannot set a lock.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3955457946
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **Future Evolution (With Versioning)**: When S3 Versioning is supported, Object Lock and Versioning will be tightly coupled following AWS S3 semantics: once Object Lock is enabled, Versioning cannot be suspended or disabled.
+> * **Bucket Creation Semantics**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+Preliminary assessments indicate that the additional metadata lookups impose an acceptable impact on overall system throughput and latency. Metrics will be continuously monitored and tuned as necessary.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions. Dedicated Access Types (BYPASS_GOVERNANCE, PUT_LEGAL_HOLD, etc.) enforce the Principle of Least Privilege, preventing unauthorized tampering or removal of locks and enhancing data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+Because this design alters the schemas of Bucket Table and Key Table, OM (Ozone Manager) version leveling will be introduced to maintain forward and backward compatibility across rolling upgrades.
Review Comment:
I think the major reason is we have added new OM APIs for the feature, so we need a new OM version to tell client whether these APIs can be called or not. And table wise, since all new fields are optional fields, they are naturally backward compatible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-6185834339
> @chungen0126 , thanks for updating the design. I have left a few inline comments.
>
> Besides, I would suggest support native ACL too, due to
>
> * some community partners(such as Shopee) use native ACL only
> * native ACL implementation is simple and straightforward
> * native ACL can be used to test the major logic easily without Ranger, so major logic testing and Ranger implementation can be proceeded in parallel.
Currently, we reuse the existing access types (READ_ACL/WRITE_ACL). If anyone prefers introducing dedicated access types specifically for Object Lock, please feel free to raise it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279635243
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
Review Comment:
Updated. The Ozone administrator role (ozone.administrators / ozone.administrators.groups) is now used with native ACLs: bypassing Governance retention is limited to Ozone administrators, because bucket and key owners have WRITE_ACL by default and could otherwise remove the Governance protection from their own data. With Ranger there is no administrator role, so the bypass is granted through Ranger policies instead. The legal hold is removed by a principal allowed to change it: WRITE_ACL on the key, plus the PutObjectLegalHold action when Ranger action matching applies (S3 Gateway with both flags enabled).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266780584
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
Review Comment:
Other operations, such as HEAD, put/get tag are also supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4291021084
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,589 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets, with authorization through Ozone native ACLs or Apache Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in Ozone is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design supports both Ozone native ACLs and Apache Ranger to manage Object Lock permissions on OBS buckets.
+Native ACLs are used by deployments without Ranger and allow the core logic to be implemented and tested independently of the Ranger integration.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, with permission and access control through native ACLs or Ranger.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Value-Level Ranger Authorization: a payload-aware Ranger condition evaluator (for example, allowing a legal hold to be set `ON` but not `OFF`) is deferred to a future phase.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Authorization: Ozone has no root account.
+ * With native ACLs, changing Object Lock settings requires `WRITE_ACL` and bypassing Governance retention is limited to Ozone administrators (see [Native ACL Mapping](#1-native-acl-mapping)).
+ * With Ranger, AWS-style action-level authorization for S3 requests requires both `ozone.s3g.sts.http.enabled` and `enableActionMatcherInPoliciesCondition` to be enabled; both are `false` by default (see [Prerequisite for S3 Fine-Grained Authorization](#3-prerequisite-for-s3-fine-grained-authorization)).
+ * Otherwise, and for native clients, Ranger falls back to `READ_ACL` / `WRITE_ACL`, and bypassing Governance retention requires `WRITE_ACL`, `WRITE` and `DELETE` (see [Fallback: Base Permission Mode](#4-fallback-base-permission-mode)).
+ * A Compliance retention cannot be removed through any API; it can only be changed offline with `ozone repair` (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until a principal allowed to change the legal hold (`WRITE_ACL` on the key, plus the `PutObjectLegalHold` action in Ranger Fine-Grained Mode) explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+* **Usage**:
+ * Used to preserve data for litigation, investigations or audits, when it is not known in advance how long the data must be kept.
+ * A legal hold has no mode and no expiration date. It is independent of retention: a key can have a legal hold, a retention period, or both,
+ and it stays locked until the legal hold is removed and the retention period has expired.
+ * A legal hold can be placed on a key in an Object Lock enabled bucket even if the key has no retention.
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * The bucket default can have both `rule` and `eventHold`. In that case a new key gets `retentionDate` = creation time + `rule` and an active event hold;
+ the key stays locked until the event hold is released and `retentionDate` has passed. With only `eventHold`, `retentionDate` is unset until the hold is released.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
Review Comment:
This is slightly contradicting what AWS provides. Consider a contract that must remain protected until closure. While it is still active, someone updates its minimum retention date. By this design, it say that says that `PutObjectRetention` on a key with an active event hold clears that hold. Protection could consequently end too early.
AWS now provides an explicit `EventHold=OFF` field. Ozone should distinguish:
- Updating retention while leaving the event hold active.
- Explicitly releasing the event hold.
- Setting or changing event-hold configuration.
AWS also makes `RetainUntilDate` optional, so release should not necessarily require the caller to supply a date. The server can calculate it from the stored duration. [AWS PutObjectRetention (https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectRetention.html)](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectRetention.html)
Could we provide an explicit event-hold release field? A normal `PutObjectRetention` update should preserve an active event hold unless release is requested.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,589 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets, with authorization through Ozone native ACLs or Apache Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in Ozone is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design supports both Ozone native ACLs and Apache Ranger to manage Object Lock permissions on OBS buckets.
+Native ACLs are used by deployments without Ranger and allow the core logic to be implemented and tested independently of the Ranger integration.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, with permission and access control through native ACLs or Ranger.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Value-Level Ranger Authorization: a payload-aware Ranger condition evaluator (for example, allowing a legal hold to be set `ON` but not `OFF`) is deferred to a future phase.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Authorization: Ozone has no root account.
+ * With native ACLs, changing Object Lock settings requires `WRITE_ACL` and bypassing Governance retention is limited to Ozone administrators (see [Native ACL Mapping](#1-native-acl-mapping)).
+ * With Ranger, AWS-style action-level authorization for S3 requests requires both `ozone.s3g.sts.http.enabled` and `enableActionMatcherInPoliciesCondition` to be enabled; both are `false` by default (see [Prerequisite for S3 Fine-Grained Authorization](#3-prerequisite-for-s3-fine-grained-authorization)).
+ * Otherwise, and for native clients, Ranger falls back to `READ_ACL` / `WRITE_ACL`, and bypassing Governance retention requires `WRITE_ACL`, `WRITE` and `DELETE` (see [Fallback: Base Permission Mode](#4-fallback-base-permission-mode)).
+ * A Compliance retention cannot be removed through any API; it can only be changed offline with `ozone repair` (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until a principal allowed to change the legal hold (`WRITE_ACL` on the key, plus the `PutObjectLegalHold` action in Ranger Fine-Grained Mode) explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+* **Usage**:
+ * Used to preserve data for litigation, investigations or audits, when it is not known in advance how long the data must be kept.
+ * A legal hold has no mode and no expiration date. It is independent of retention: a key can have a legal hold, a retention period, or both,
+ and it stays locked until the legal hold is removed and the retention period has expired.
+ * A legal hold can be placed on a key in an Object Lock enabled bucket even if the key has no retention.
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * The bucket default can have both `rule` and `eventHold`. In that case a new key gets `retentionDate` = creation time + `rule` and an active event hold;
+ the key stays locked until the event hold is released and `retentionDate` has passed. With only `eventHold`, `retentionDate` is unset until the hold is released.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the latest of the existing `retentionDate`, the requested `RetainUntilDate`, and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **authorization (native ACLs or Ranger)** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Base Permissions**: Object Lock operations are checked against the existing `READ_ACL` / `WRITE_ACL` types for both native ACLs and Ranger, so no new ACL type is needed (see below).
+* **Fine-Grained S3 Action Matching (Ranger)**: For requests through S3 Gateway, Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Native ACL Mapping
+
+Changing Object Lock state is treated as a permission change, so it requires `WRITE_ACL` rather than `WRITE`.
+A user who can only write or delete data cannot remove or shorten a lock.
+
+| Operation | Resource Level | Required Native ACL |
+|:---------------------------------------|:---------------|:-----------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ_ACL` |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE_ACL` |
+| `GetObjectRetention` | Key | `READ_ACL` |
+| `PutObjectRetention` | Key | `WRITE_ACL` |
+| `GetObjectLegalHold` | Key | `READ_ACL` |
+| `PutObjectLegalHold` | Key | `WRITE_ACL` |
+| Bypass Governance retention | Key | Ozone administrator only |
+
+**Ozone administrator** means a user listed in `ozone.administrators` or a member of a group listed in `ozone.administrators.groups`
+(if neither is set, only the user that starts the OM is an administrator). This is the same check used for other admin-only OM operations.
+* The native authorizer already grants every ACL to Ozone administrators, so they also pass the `READ_ACL` / `WRITE_ACL` checks above.
+* Bypass is limited to administrators because the native ACL model has no permission dedicated to it: any ACL type would also be grantable by bucket or key owners,
+ who could then remove the Governance protection from their own data.
+* Read-only administrators (`ozone.readonly.administrators`) and S3 administrators (`ozone.s3.administrators`) cannot bypass Governance retention.
+* The administrator role is cluster wide, so it cannot be scoped to a single volume or bucket. Deployments that need per-bucket bypass entitlement should use Ranger.
+* The bypass does not apply to Compliance retention or a legal hold; a legal hold must be removed explicitly with `PutObjectLegalHold`, which an administrator can do because it only requires `WRITE_ACL`.
+
+---
+
+#### 2. Ranger Authorization Modes
+
+With Ranger, every Object Lock request is authorized in one of two modes.
+S3 actions are an S3 protocol concept, so only S3 Gateway sets them; OM does not add S3 actions to requests itself.
+In both modes OM always checks the base access type (`READ_ACL` / `WRITE_ACL`), and the OM WORM state validation is the same.
+
+| Request source | `ozone.s3g.sts.http.enabled` | `enableActionMatcherInPoliciesCondition` | Mode |
+|:--------------------------------|:-----------------------------|:-----------------------------------------|:------------------------------------------------------------------------|
+| S3 Gateway | `true` | `true` | [Fine-Grained Mode](#3-prerequisite-for-s3-fine-grained-authorization) |
+| S3 Gateway | `false` | `true` | [Base Permission Mode](#4-fallback-base-permission-mode); denied by policy items with an action condition (see below) |
+| S3 Gateway | `false` | `false` | [Base Permission Mode](#4-fallback-base-permission-mode) |
+| S3 Gateway | `true` | `false` | [Base Permission Mode](#4-fallback-base-permission-mode) (not recommended, see below) |
+| Native Java API / CLI | any | any | [Base Permission Mode](#4-fallback-base-permission-mode); with the Ranger flag enabled, denied by policy items with an action condition |
+
+* **Fine-Grained Mode**: Ranger checks the S3 action (for example `PutObjectLegalHold`) and its base access type in the [mapping below](#6-action-to-ranger-access-type-mapping).
+* **Base Permission Mode**: Ranger checks only the base access type.
+
+**Requests without an action when the Ranger flag is enabled**:
+
+When `enableActionMatcherInPoliciesCondition=true`, the result for a request without an S3 action (S3 Gateway with the STS flag disabled, or a native client) depends on the matching Ranger policy item:
+
+| Matching policy item | Request with an S3 action | Request without an S3 action |
+|:----------------------------------------------|:---------------------------------------------------|:-----------------------------------------------------|
+| Has an action condition (e.g. `PutObjectLegalHold`) | Granted only if the action is listed and the access type matches | **Denied**: the action condition cannot be satisfied |
+| Has no action condition | Granted if the access type matches | Granted if the access type matches (Base Permission Mode) |
+
+* If the principal is granted Object Lock access only through policy items with an action condition, every Object Lock request without an action is denied,
+ including the requests that the same principal could send through S3 Gateway with both flags enabled.
+* If the principal is granted the access type through a policy item without an action condition, the request falls back to Base Permission Mode.
+* When `enableActionMatcherInPoliciesCondition=false`, action conditions are not evaluated at all, so every request is in Base Permission Mode.
+
+---
+
+#### 3. Prerequisite for S3 Fine-Grained Authorization
+
+> _**Important for S3 users**_: AWS-style action-level authorization for Object Lock (separate permissions for `PutObjectRetention`,
+> `PutObjectLegalHold`, `PutBucketObjectLockConfiguration` and `BypassGovernanceRetention`) is only available when **both** flags below are enabled.
+> Both are `false` by default, so a cluster with default settings authorizes S3 Object Lock requests in [Base Permission Mode](#4-fallback-base-permission-mode).
+
+```properties
+# S3 Gateway and OM: extract the S3 action and forward it to the authorizer
+ozone.s3g.sts.http.enabled=true
+# Ranger Ozone service definition: evaluate the S3 action in policy conditions
+ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+```
+
+**Why both flags are required**:
+* `ozone.s3g.sts.http.enabled` decides whether the S3 action reaches Ranger at all. Today S3 Gateway sets `S3Authentication.s3Action` and OM copies it into the `RequestContext` only when this flag is `true`
+ (`EndpointBase.applyS3Action`, `OmMetadataReader.maybeAddToContextFromThreadLocal`). The action was originally only needed for STS, which is why it is tied to this flag.
+* `enableActionMatcherInPoliciesCondition` decides whether Ranger uses the action. When it is `false`, Ranger's `RangerActionMatcher` ignores the action and only checks the access type.
+* If only the Ranger flag is enabled, S3 Gateway sends no action. Requests are denied by policy items with an action condition, and fall back to Base Permission Mode only through policy items without one
+ (see [Requests without an action when the Ranger flag is enabled](#2-ranger-authorization-modes)). S3 users granted Object Lock permissions by action would therefore be denied, so this combination is not recommended either.
+* If only the STS flag is enabled, Ranger ignores the action, so requests also fall back to Base Permission Mode.
+ This combination is not recommended: STS session policies restrict temporary credentials by S3 action (see [AWS STS Design for Ozone S3](ozone-sts.md)),
+ so without the Ranger flag those restrictions would not be enforced. This is also why the STS flag is not enabled by default.
+
+**What S3 users get in Fine-Grained Mode**:
+* Object Lock permissions can be granted separately, as in AWS IAM. For example, a legal team can be granted `PutObjectLegalHold` without `PutObjectRetention`,
+ and `BypassGovernanceRetention` can be granted to a smaller group than the one that sets retention.
+* STS session policies (AssumeRole) can include or exclude Object Lock actions, and `IamSessionPolicyResolver` restricts the temporary credential accordingly.
+* Each action still requires its base access type, so a wrong action mapping in S3 Gateway cannot reduce the check below `READ_ACL` / `WRITE_ACL`.
+
+**Operator guidance**:
+* Clusters that expose Object Lock to S3 users with different roles (for example, data writers, compliance officers and administrators) should enable both flags before enabling Object Lock on any bucket.
+* The STS flag only affects S3 Gateway requests. Native clients never carry an action, so they are denied by policy items with an action condition and are granted only through policy items without one (see [below](#4-fallback-base-permission-mode)).
+* The user documentation of this feature will describe both flags and the difference between the two modes.
+
+---
+
+#### 4. Fallback: Base Permission Mode
+
+Base Permission Mode is used for every request without an S3 action:
+requests from the native Java API or CLI, and S3 Gateway requests when either flag in the [previous section](#3-prerequisite-for-s3-fine-grained-authorization) is disabled.
+OM does not set an S3 action for these requests, so that the S3 action mapping stays in `S3GActionIamMapper` only.
+When the Ranger flag is enabled, these requests are granted only through policy items without an action condition
+(see [Requests without an action when the Ranger flag is enabled](#2-ranger-authorization-modes)).
+
+**Checks**:
+
+| Operation | Resource Level | Required Ranger Access Type |
+|:---------------------------------------|:---------------|:----------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ_ACL` |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE_ACL` |
+| `GetObjectRetention` | Key | `READ_ACL` |
+| `PutObjectRetention` | Key | `WRITE_ACL` |
+| `GetObjectLegalHold` | Key | `READ_ACL` |
+| `PutObjectLegalHold` | Key | `WRITE_ACL` |
+| Bypass Governance retention | Key | `WRITE_ACL`, `WRITE` and `DELETE` |
+
+**Why `READ_ACL` / `WRITE_ACL` instead of `READ` / `WRITE`**:
+* Without an S3 action, a `WRITE` check could not tell `PutObject` from `PutObjectLegalHold`, so any user who can write a key could also remove its legal hold or shorten its retention.
+* Changing a lock is treated as a permission change, so it uses the same access type as changing an ACL. Users with only `READ`, `WRITE`, `CREATE` or `DELETE` on a key cannot read or change its Object Lock settings.
+* This matches the [Native ACL Mapping](#1-native-acl-mapping), so the same operation needs the same access type with native ACLs and with Ranger.
+
+**Limitations compared with Fine-Grained Mode**:
+* All Object Lock changes on a resource need the same `WRITE_ACL`, so they cannot be granted separately. A principal that can set a legal hold can also set retention on the same key, and can bypass Governance retention if it also has `WRITE` and `DELETE`.
+* Governance retention therefore only protects keys from principals without `WRITE_ACL`, `WRITE` and `DELETE`. Deployments that need a separate bypass entitlement should use Fine-Grained Mode.
+* `WRITE_ACL` is also the access type for changing ACLs, so granting Object Lock management also grants that.
+* Unlike native ACLs, where bypass is limited to Ozone administrators, Ranger has no Ozone administrator role, so bypass is granted through a policy with `WRITE_ACL`, `WRITE` and `DELETE`.
+* Compliance retention is not affected: it cannot be removed in either mode.
+
+**Native clients when both flags are enabled**:
+* Native clients carry no S3 action. A policy item with an action condition cannot be satisfied by a request without an action, so it does not grant the request.
+* A principal whose `WRITE_ACL` is limited to `PutObjectLegalHold` through S3 Gateway is therefore denied all Object Lock operations through a native client;
+ it cannot use a native client to get around the action-level restrictions.
+* To let a principal manage Object Lock through a native client, grant the access type through a policy item without an action condition.
+ That item applies to S3 Gateway requests as well, so the principal gets every Object Lock operation allowed by the access type, without action-level separation.
+
+---
+
+#### 5. Ranger & S3 Gateway Component Updates
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (`READ_ACL`, `WRITE_ACL`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings, mapped to the same `READ_ACL` / `WRITE_ACL` base access types.
+
+---
+
+#### 6. Action to Ranger Access Type Mapping
+
+In Fine-Grained Mode, following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ_ACL` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE_ACL` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ_ACL` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE_ACL` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ_ACL` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE_ACL` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE_ACL`, `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 7. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable through the Ozone API. **No role, administrator, native ACL or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires. The only way to change it early is the offline `ozone repair` command described below.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Bypass Entitlement**: With native ACLs, the principal must be an Ozone administrator (`ozone.administrators`).
+ With Ranger, the principal needs `WRITE_ACL`, `WRITE` and `DELETE` on the target key, and in Fine-Grained Mode also the **`BypassGovernanceRetention`** action.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted through the Ozone API by any admin, native ACL or Ranger policy until the retention period expires.
+> It can only be changed offline with `ozone repair` (see below).
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+* Bucket links: S3 buckets are often links (for example `/s3v/bucket1` → `/vol1/bucket1`). Unlike `OMBucketSetPropertyRequest`, which rejects property changes on links,
+ `PutBucketObjectLockConfiguration` follows the link and stores the configuration on the source bucket, and `GetBucketObjectLockConfiguration` reads it from there.
+ Authorization is checked on both the link and the source bucket, like other requests that resolve links. Key operations already resolve links, so WORM checks always use the source bucket's configuration.
+
+**Object-Level APIs:**
Review Comment:
Below APIs doing and implicitly assuming permissions like updating retention date along with upload. Do we have a clear permission matrix ?
For example: a client application writing data should not gain permission to choose a shorter retention period merely by adding a retention header to an upload. Equally, permission to manage retention should not grant permission to upload data.
[AWS permission requirements (https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3963526702
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
Review Comment:
hi all - you won't need to create new access types for this. From the STS project, the concept of actions are created in Ranger, which map exactly to the AWS actions without the `s3:` prefix. From the AWS documentation (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html), there are 6 new actions:
```
s3:BypassGovernanceRetention
s3:GetBucketObjectLockConfiguration
s3:GetObjectLegalHold
s3:GetObjectRetention
s3:PutBucketObjectLockConfiguration
s3:PutObjectLegalHold
s3:PutObjectRetention
```
so in Ranger these would be:
```
BypassGovernanceRetention
GetBucketObjectLockConfiguration
GetObjectLegalHold
GetObjectRetention
PutBucketObjectLockConfiguration
PutObjectLegalHold
PutObjectRetention
```
For example, if you want the role to have access `PutObjectLegalHold`, you would grant `PutObjectLegalHold` action and `write` permission (i.e. access type) in Ranger. Or you could grant `Put*` and `all` permission and get access to all the Put actions.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
Review Comment:
The mappings in Ranger for actions to permissions/access types are found here: https://github.com/apache/ranger/blob/master/security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json.
Also, please note that when STS is merged to master, the `IamSessionPolicyResolver` would need to get updated with the new actions as well as `S3GActionIamMapper`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3963526702
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
Review Comment:
hi all - you won't need to create new access types for this. From the STS project, the concept of actions are created in Ranger, which map exactly to the AWS actions without the `s3:` prefix. From the AWS documentation (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html), there are 7 new actions:
```
s3:BypassGovernanceRetention
s3:GetBucketObjectLockConfiguration
s3:GetObjectLegalHold
s3:GetObjectRetention
s3:PutBucketObjectLockConfiguration
s3:PutObjectLegalHold
s3:PutObjectRetention
```
so in Ranger these would be:
```
BypassGovernanceRetention
GetBucketObjectLockConfiguration
GetObjectLegalHold
GetObjectRetention
PutBucketObjectLockConfiguration
PutObjectLegalHold
PutObjectRetention
```
For example, if you want the role to have access `PutObjectLegalHold`, you would grant `PutObjectLegalHold` action and `write` permission (i.e. access type) in Ranger. Or you could grant `Put*` and `all` permission and get access to all the Put actions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4345196339
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
Here "Ranger has no administrator role", but OM's Ozone administrator check (`ozone.administrators`) doesn't depend on the authorizer. OM already uses it on Ranger clusters, for example `OMSnapshotCreateRequest` only allows an Ozone admin or the bucket owner to create snapshots.
This matters because with default Ranger settings (`action-matches` off), any user with `WRITE` on `bucket1/*` can set `COMPLIANCE` with a date 100 years out on every key in the bucket. Nobody can undo that through the API; the only fix is stopping all OMs and running the offline repair.
Could Compliance changes need an Ozone administrator in both models, the same as with native ACLs?
Just this clarification I need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4313463230
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
Review Comment:
@ChenSammi @fmorg-git Updated the doc:
**Native ACLs** (following @ChenSammi's proposal):
- `GetBucketObjectLockConfiguration` / `GetObjectRetention` / `GetObjectLegalHold` require `READ`.
- Changing a legal hold, a Governance retention, or the bucket configuration is limited to the owner of the volume or bucket (the same owner the native authorizer already lets skip ACL checks), or an Ozone administrator. The owner can also bypass Governance retention.
- Setting a Compliance retention is limited to Ozone administrators: setting it on a key (`PutObjectRetention` or the `x-amz-object-lock-*` headers), changing Governance to Compliance, and setting Compliance as a bucket default. Keys that inherit a Compliance bucket default do not need an extra check. Extending a Compliance retention or releasing its event hold only needs the owner, as in AWS S3 where it only needs `PutObjectRetention`.
**Ranger**:
- The doc now describes two authorization models, native ACLs and Ranger, and recommends enabling `enableActionMatcherInPoliciesCondition` for fine-grained permissions. I removed the STS flag combinations and the "fallback" wording.
- The doc depends on HDDS-17054, so S3 Gateway always sends the S3 action.
- @fmorg-git, on the concern about actions being set in several places: the new OM requests do not set S3 actions.
- With Ranger, OM maps each operation to an access type, and `action-matches` only narrows a policy item. Requests without an S3 action (ofs, o3fs, CLI, native Java API) are matched only by items without the condition or with `*`, so switching to a non-S3 client cannot get around an action restriction. This is now described in the Trusted Boundary section; deployments with both S3 and non-S3 access should keep such items for non-S3 clients.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4357768649
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
Review Comment:
I believe the API name is GetObjectLockConfiguration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4358063413
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Setting Compliance only needs `PutObjectRetention` (`WRITE`), as in AWS S3, so it can be delegated per bucket with Ranger policies.
+ Requiring an Ozone administrator, as with native ACLs, would make it cluster wide.
+ Only `action-matches` can limit who sets it. No Ranger permission can remove a Compliance retention; only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+##### Access Type Mapping
+
+Policy items without `action-matches` are checked by access type only.
+
+| S3 Action | Resource | Access type | Description |
+|:-----------------------------------|:---------|:---------------------|:----------------------------------------------------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` | Read the bucket's Object Lock settings. |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE` | Set the bucket's default retention (mode, period, event hold). |
+| `GetObjectRetention` | Key | `READ` | Read the retention mode, date and event hold. |
+| `PutObjectRetention` | Key | `WRITE` | Set, extend, shorten or remove a retention; set or release an event hold. |
+| `GetObjectLegalHold` | Key | `READ` | Read the legal hold. |
+| `PutObjectLegalHold` | Key | `WRITE` | Set the legal hold `ON` or `OFF`. |
+| `BypassGovernanceRetention` | Key | `WRITE` and `DELETE` | Bypass Governance retention. |
+
+##### `action-matches`
+
+* With `enableActionMatcherInPoliciesCondition`, a policy item can have an `action-matches` condition: a list of S3 actions (`*` and `Prefix*` allowed)
+ that `RangerActionMatcher` compares with the action of the request.
+* It only narrows an item: the item applies when both the access type and the action match. Items without it behave as before.
+* Example: grant a legal team `WRITE` with `action-matches: PutObjectLegalHold`, so it can change legal holds but not retention.
+* S3 Gateway sets the action in `S3GActionIamMapper`, and OM passes it to Ranger in the `RequestContext`. OM only adds the dependent actions below.
+* Today S3 Gateway only sends the action when `ozone.s3g.sts.http.enabled=true`. This design depends on [HDDS-17054](https://issues.apache.org/jira/browse/HDDS-17054), which sends it for every S3 request.
+* For requests without an S3 action (ofs, o3fs, Ozone CLI, Java API), see [Trusted Boundary](#trusted-boundary-and-compliance-scope).
Review Comment:
Can we put the conclusion here, that it will fail, besides the reference to Trusted Boundary.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Setting Compliance only needs `PutObjectRetention` (`WRITE`), as in AWS S3, so it can be delegated per bucket with Ranger policies.
+ Requiring an Ozone administrator, as with native ACLs, would make it cluster wide.
+ Only `action-matches` can limit who sets it. No Ranger permission can remove a Compliance retention; only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+##### Access Type Mapping
+
+Policy items without `action-matches` are checked by access type only.
+
+| S3 Action | Resource | Access type | Description |
+|:-----------------------------------|:---------|:---------------------|:----------------------------------------------------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` | Read the bucket's Object Lock settings. |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE` | Set the bucket's default retention (mode, period, event hold). |
+| `GetObjectRetention` | Key | `READ` | Read the retention mode, date and event hold. |
+| `PutObjectRetention` | Key | `WRITE` | Set, extend, shorten or remove a retention; set or release an event hold. |
+| `GetObjectLegalHold` | Key | `READ` | Read the legal hold. |
+| `PutObjectLegalHold` | Key | `WRITE` | Set the legal hold `ON` or `OFF`. |
+| `BypassGovernanceRetention` | Key | `WRITE` and `DELETE` | Bypass Governance retention. |
+
+##### `action-matches`
+
+* With `enableActionMatcherInPoliciesCondition`, a policy item can have an `action-matches` condition: a list of S3 actions (`*` and `Prefix*` allowed)
+ that `RangerActionMatcher` compares with the action of the request.
+* It only narrows an item: the item applies when both the access type and the action match. Items without it behave as before.
+* Example: grant a legal team `WRITE` with `action-matches: PutObjectLegalHold`, so it can change legal holds but not retention.
+* S3 Gateway sets the action in `S3GActionIamMapper`, and OM passes it to Ranger in the `RequestContext`. OM only adds the dependent actions below.
+* Today S3 Gateway only sends the action when `ozone.s3g.sts.http.enabled=true`. This design depends on [HDDS-17054](https://issues.apache.org/jira/browse/HDDS-17054), which sends it for every S3 request.
+* For requests without an S3 action (ofs, o3fs, Ozone CLI, Java API), see [Trusted Boundary](#trusted-boundary-and-compliance-scope).
Review Comment:
Can we put the direct conclusion here, that it will fail, besides the reference to Trusted Boundary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4397427029
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
Review Comment:
Agreed. Based on our previous discussion, offline repair is essentially equivalent to requiring administration, so I will remove the Offline Compliance Retention Repair section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3987576472
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+
+### Impact on Write and Delete Flows
+#### Impact on Put Object Flow
+
+Put Flow:
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+#### Impact on Multipart Upload Flow
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. During Initiate Multipart Upload and Put Part phases, WORM checks are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. During the Complete Multipart Upload phase, final validation occurs in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+#### Impact on Delete Object Flow
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+Preliminary assessments indicate that the additional metadata lookups impose an acceptable impact on overall system throughput and latency. Metrics will be continuously monitored and tuned as necessary.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions. Dedicated Access Types (BYPASS_GOVERNANCE, PUT_LEGAL_HOLD, etc.) enforce the Principle of Least Privilege, preventing unauthorized tampering or removal of locks and enhancing data immutability.
Review Comment:
Thanks for pointing this out!
I've added a Trusted Boundary section to clarify that any operations requiring root access bypass the application-level API and are outside the trusted boundary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3955528607
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
Review Comment:
Do we have an JIRA open in Ranger project to track all the changes required for this feature?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3954820020
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **Future Evolution (With Versioning)**: When S3 Versioning is supported, Object Lock and Versioning will be tightly coupled following AWS S3 semantics: once Object Lock is enabled, Versioning cannot be suspended or disabled.
+> * **Bucket Creation Semantics**:
Review Comment:
There is no more S3 behavior, versioning is enabled automatically when Object Lock is enabled.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **Future Evolution (With Versioning)**: When S3 Versioning is supported, Object Lock and Versioning will be tightly coupled following AWS S3 semantics: once Object Lock is enabled, Versioning cannot be suspended or disabled.
+> * **Bucket Creation Semantics**:
Review Comment:
There is one more S3 behavior, versioning is enabled automatically when Object Lock is enabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-5687106351
> @chungen0126 , just want to highlight to be in compliance with AWS S3 WORM, that AWS has recently added variable retention with event holds to S3 Object Lock, so may be we can think about adding EVENT_TRIGGERED_RETENTION in our existing enum for extended support. Enterprise S3 semantics are increasingly about governance workflows, not just object durability. Reference : [AWS S3 doc](https://aws.amazon.com/about-aws/whats-new/2026/09/amazon-s3-object-lock-variable-retention/)
>
> ```
> object created
> |
> event hold
> |
> business event occurs
> |
> hold released
> |
> retention timer starts
> |
> WORM until timer expires
> ```
Thanks @devmadhuu for the reminder. I add a sub-section in Terminology to introduce the trigger workflow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4022161741
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+
+### Impact on Write and Delete Flows
+#### Impact on Put Object Flow
+
+Put Flow:
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+#### Impact on Multipart Upload Flow
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. During Initiate Multipart Upload and Put Part phases, WORM checks are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. During the Complete Multipart Upload phase, final validation occurs in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+#### Impact on Delete Object Flow
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
Review Comment:
This can be divided into two parts:
1. **Bucket or volume removal:**
Currently, a bucket cannot be deleted if it still contains keys. Therefore, this is not impacted.
2. **Directory removal:**
Regarding the non-goal, the current design does not support FSO or legacy buckets, so it will not touch upon directory removal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4321114332
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
+one by one from the client, so it is blocked by Object Lock as well. Ozone has no account concept, so unlike AWS
+(where closing the account destroys all data, including Compliance-locked objects), there would be no way to decommission
+a tenant or volume until every retention period expires. An admin-only command is provided for this case,
+with the volume as the unit of deletion, as the closest analogue to deleting an AWS account:
+
+```text
+ozone admin om volume force-delete <volume>
+```
+
+* **Disabled by default**: guarded by a new OM config (for example `ozone.om.object.lock.volume.force.delete.enabled`, default `false`).
+ Deployments that must never allow removal of Compliance-locked data keep it disabled.
+* **Authorization**: OM administrators only (`ozone.administrators`). It is not exposed through S3 Gateway and is not covered by `BypassGovernanceRetention`.
+* **Execution**: OM marks the volume as deleting and rejects any new request on it. A background service then purges buckets and keys in batches
+ (including open keys, multipart uploads and snapshots), because a single Ratis transaction cannot delete a large volume. WORM checks are skipped only for this purge.
+ Deletion is irreversible.
+* **Audit**: the request and the purge result are written to the OM audit log.
+* **Trusted boundary**: this is the only way to remove Compliance-locked data through the Ozone API. The data is protected from everyone except a cluster administrator who explicitly enabled and ran this command.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+The additional metadata lookups are expected to have a small impact on throughput and latency; this will be verified with benchmarks and metrics during implementation.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions.
+
+By leveraging the **STS action-matching framework**, Ozone maps standard S3 API actions (e.g., `BypassGovernanceRetention`, `PutObjectLegalHold`) directly to standard Ranger primitive access types (e.g., `WRITE` and `READ`). This fine-grained, action-level authorization enforces the Principle of Least Privilege **without requiring custom dedicated access types**. It effectively prevents unauthorized tampering or removal of locks, ensuring robust compliance and data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+* **System Clock Tampering**: Artificially manipulating or rolling back the system clock (NTP/OS time) on OM or DataNode hosts to prematurely expire object retention periods or evade time-based WORM validation.
+* **Ranger Super Admin Privilege Abuse**: Modifying, disabling, or overriding Ozone authorization policies via Apache Ranger admin credentials (or direct access to Ranger’s backend database), allowing privileged operators to alter bucket WORM flags or grant unrestricted deletion permissions outside normal enforcement workflows.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+* **Table Schema Changes**: Since all newly introduced fields in the Bucket Table (`objectLockEnabled`, `defaultRetention`) and Key Table (`retention`, `legalHold`) are defined as `optional` in Protobuf, they are wire-compatible.
+ To avoid RocksDB serialization problems if the cluster is rolled back after new fields were written, a new OM layout feature gates Object Lock: until the upgrade is finalized, OM rejects every Object Lock request (bucket configuration, retention, legal hold, and `x-amz-object-lock-*` headers on Put / Multipart Upload), so no Object Lock metadata is persisted before finalization.
+* **OM Version Leveling**: OM version leveling is introduced primarily because this feature adds new OM APIs (such as `setRetentionConfig` and `setLegalHold`). The new OM version allows clients to determine whether the target OM cluster supports executing these Object Lock APIs during rolling upgrades.
+
+## Alternatives
+
+### 1. Why Not Use Native ACLs for Object Lock Management?
+
+* **Proposal**:
+ Support Native ACLs alongside or instead of Ranger to configure and manage Object Lock states (e.g., using Ozone native ACLs or POSIX-like ACLs at the Volume/Bucket/Key level).
+
+* **Why Rejected**:
+ 1. **Conflicting Security & Authorization Philosophies**:
+ Native ACLs in Ozone are primarily designed for granular, resource-centric permissions (often granting permissions directly to specific objects/buckets or inherited via path hierarchies). In contrast, Object Lock is an enterprise compliance feature that requires strict, role-based access governance (RBAC/ABAC). Compliance controls like WORM and Legal Holds are inherently tied to compliance officers, security auditors, or legal teams—not individual data owners.
+ 2. **Risk of Decentralized & Inconsistent Governance**:
+ Native ACLs allow object creators or bucket owners to adjust permissions. If lock controls were exposed through Native ACLs, standard bucket administrators or key owners could modify or override lock-related ACLs locally. Centralizing lock authorization exclusively in **Apache Ranger** ensures that compliance policies are managed from a unified pane of glass, preventing local policy drift or accidental misconfigurations.
+ 3. **Operational Complexity**:
+ Supporting dual authorization models (Ranger + Native ACLs) for critical compliance features introduces unnecessary complexity, potential security loopholes, and ambiguous resolution hierarchies during audits. Excluding Native ACLs ensures clean boundary enforcement.
+
+## Plan
+
+The implementation of S3 Object Lock in Apache Ozone is divided into six structured phases. This ensures incremental delivery, clean service boundaries, and testability at each stage.
+
+### Phase 1: Metadata Infrastructure & Schema Definitions
+
+*Goal: Establish protobuf schemas, in-memory domain models, and RocksDB table serialization codecs.*
+
+1. **Protobuf Definitions (`OmClientProtocol.proto`)**:
+ - Introduce retention domain messages:
+ - `RetentionMode` enum (`GOVERNANCE = 1`, `COMPLIANCE = 2`).
+ - `Rule` message (`retentionMode`, `timeUnit`, `duration`).
+ - `EventHold` message (`enabled`, `rule`).
+ - `RetentionConfig` message (`retentionMode`, `rule`, `eventHold`).
+ - `Retention` message (`retentionMode`, `retentionDate`, `eventHold`).
+ - Extend `BucketInfo` (OBS buckets only):
+ - `optional bool objectLockEnabled = 24 [default = false];`
+ - `optional RetentionConfig defaultRetention = 25;`
+ - Extend `KeyInfo` & `KeyInfoProtoLight`:
+ - `optional Retention retention = 23;`
+ - `optional bool legalHold = 24 [default = false];`
+
+2. **Domain Models & Helpers (`hadoop-ozone/common`)**:
+ - Update `OmBucketInfo` / `OmBucketInfo.Builder` with getters, setters, and protobuf translation.
+ - Update `OmKeyInfo` / `OmKeyInfo.Builder` with getters, setters, and protobuf translation.
+ - Introduce new class `Retention` to encapsulate bucket rules of retention:
+ - `fromProto` to convert from protobuf to java class.
+ - `toProto` to convert from java class to protobuf.
+ - Introduce `RetentionUtil` to provide utility methods for retention calculations:
+ - Check if `retention` (`retentionDate`, `eventHold`) is still locking: a key is locked if `legalHold` is true, `eventHold.enabled` is true, or `retentionDate` is later than the request time stamped by the leader in `preExecute` (not the local clock of each OM).
+ - Validate retention rules (e.g., duration > 0, valid time unit, etc.).
+
+---
+
+### Phase 2: Ranger Integration & Action Mapping
+
+*Goal: Enable fine-grained access control separating standard data mutating permissions from compliance actions.*
+
+1. **Ranger Action Mapping & UI Validation (`ozone.json`)**:
+ - Do **NOT** create new underlying Ranger access types. Retain primitive access types (`READ`, `WRITE`, `DELETE`) and leverage the STS action-matching framework (`action-matches`).
+ - Register the 7 S3 Object Lock actions in `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json`:
+ - `GetBucketObjectLockConfiguration` (Bucket level: requires `READ`)
+ - `PutBucketObjectLockConfiguration` (Bucket level: requires `WRITE`)
+ - `GetObjectRetention` (Key level: requires `READ`)
+ - `PutObjectRetention` (Key level: requires `WRITE`)
+ - `GetObjectLegalHold` (Key level: requires `READ`)
+ - `PutObjectLegalHold` (Key level: requires `WRITE`)
+ - `BypassGovernanceRetention` (Key level: requires `WRITE` / `DELETE`)
+ - Enable design-time validation in Ranger React UI to ensure administrators cannot grant compliance actions without selecting their required primitive permissions.
+
+2. **Payload-Aware String Matcher Condition Evaluator**:
+ - Register a dedicated custom condition evaluator (e.g., `RangerOzoneValueMatcher`, extending `RangerAbstractConditionEvaluator`) in `ranger-servicedef-ozone.json` under `policyConditions`.
+ - Update `RangerOzoneAuthorizer` and request context handling to extract sensitive payload attributes and inject them into `RangerAccessRequest.getContext()`:
+ - `s3:object-lock-legal-hold`: Extracted from the request body (`ON` vs. `OFF`).
+ - `s3:object-lock-mode`: Extracted from the request body (`GOVERNANCE` vs. `COMPLIANCE`).
+ - `s3:x-amz-bypass-governance-retention`: Extracted from request headers (`true` vs. `false`).
+ - Implement runtime string matching within the evaluator to enforce value-level role segregation:
+ - Allow decoupling privileged operations (e.g., granting `PutObjectLegalHold` exclusively when `s3:object-lock-legal-hold == "ON"`, while restricting `OFF` to compliance officers).
+ - Prevent accidental or unauthorized lock-in by enforcing `s3:object-lock-mode == "GOVERNANCE"` for standard operators, denying `COMPLIANCE` configuration.
+
+3. **S3 Gateway & IAM Action Resolution**:
+ - Update `S3GAction` in `hadoop-ozone/s3gateway`: add audit actions for all lock configuration operations.
+ - Update `S3GActionIamMapper` in `hadoop-ozone/s3gateway`: map the new audit actions to standard IAM S3 actions (`GetBucketObjectLockConfiguration`, `PutBucketObjectLockConfiguration`, `GetObjectRetention`, `PutObjectRetention`, `GetObjectLegalHold`, `PutObjectLegalHold`, `BypassGovernanceRetention`).
+ - Update `IamSessionPolicyResolver.S3Action` in `hadoop-ozone/common`: register the actions with their resource scopes (Bucket vs. Object) and base permissions (`READ`, `WRITE`).
+ - Propagate `s3Action` via `S3Auth` through OM RPC to `RequestContext.s3Action` for evaluation by `RangerOzoneAuthorizer`.
Review Comment:
Follow-up on the three gaps with the latest design:
1. OM picks the access type from the request type, so a wrong label in S3 Gateway cannot lower the check below `WRITE`.
2. Native clients send no action; with Ranger they are only matched by policy items without `action-matches` (or with `*`), and with native ACLs they need the owner, or an Ozone administrator for Compliance.
3. With HDDS-17054, the action is sent even when STS is disabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4321107184
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
Review Comment:
Follow-up: the doc no longer requires the STS flag. It depends on HDDS-17054, so S3 Gateway sends the S3 action for every request
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4349812257
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
You're right, OM's administrator check does not depend on the authorizer, so "Ranger has no administrator role" was not the right reason.However, I'd still keep Compliance under Ranger policies instead of requiring an Ozone administrator:
1. A Compliance retention set by mistake is not stuck forever: `ozone repair om object-lock` can remove or downgrade it offline.
2. `ozone.administrators` is cluster wide, so anyone who manages Compliance for a single bucket would have to be an administrator of the whole cluster.
3. It would remove Ranger's fine-grained delegation, for example granting `PutObjectRetention` with `action-matches` to a compliance team on specific buckets only, which is the main reason to use Ranger for this feature.
For the risk of action matcher being off by default, we've highlighted it in the doc and will cover it in the user guide. The recommended mitigation remains enabling action-matches.
I've updated the doc. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279650884
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
Review Comment:
Renamed to `BucketRetention`. A bucket can have both `rule` and `eventHold`. When both are set, it operates as follows:
1. **Key Creation**: Its `retentionDate` is set to `now + rule`, and `event hold = ON`. The key is locked because the event hold is active (`ON`).
2. **Event Hold Release (`event hold = OFF`)**: OM calculates `now + eventHold.rule` and sets `retentionDate` to the latest of:
- `now + eventHold.rule`
- The existing `retentionDate`
- The `RetainUntilDate` specified in the release request (event holds are released through `PutObjectRetention`, which carries a `RetainUntilDate`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266918042
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
Review Comment:
It's not a good idea to support a `ozone sh volume delete -r` command, to minic the AWS account close situation. Whether and how Ozone should support an account close situation, is out of the scope of this feature.
For this feature, the real problem is if Compliance Mode is put on a object, is there anyone can remove it?
If we want to follow the AWS rule, then instead of provide the volume deletion command, provide a offline Compliance Mode update command per key or batch is more appropriate. @chungen0126 , there is this "ozone repair" command category, where commands are provided to fix the problems offline.
If we want to make it more convenient, we can have a Compliance Mode update command per key or batch, and limit it to only ozone admins.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266821265
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
Review Comment:
Let's change this to BucketRetentionConfig or BucketRetention, to be consistent and distinguish with key.
Can this have both rule and eventHold, or they are one of relationship?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266794219
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
Review Comment:
Legal hold usage is missing here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4278212472
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
Review Comment:
Done. "OBS" is now used only for the bucket type; this sentence says "Ozone".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4290230022
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
@chungen0126 , since we don't have these fine grained permissions in native ACL than Ranger, so instead of using READ_ACL or WRITE_ACL, I would like to propose,
for object and bucket
- READ permission for lock information receive
- owner for lock information update(except the compliance update action)
for compliance mode update
- Only ozone admin
And fallback is not supported, if Ranger is configured, when always use Ranger for the authorization; If Native ACL is configured, when always use Native ACL.
So if user wants the fine grained permission control, then he/she use Ranger, which is our recommendation too for this feature. If Ranger is not available, then Native ACL can be used, which provides the same(or even stricter) level of security protection.
@symious @xichen01 @ivandika3 , will this meet your requirements if in future you guys want to use this feature?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4415345132
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Setting Compliance only needs `PutObjectRetention` (`WRITE`), as in AWS S3, so it can be delegated per bucket with Ranger policies.
+ Requiring an Ozone administrator, as with native ACLs, would make it cluster wide.
+ Only `action-matches` can limit who sets it. No Ranger permission can remove a Compliance retention; only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+##### Access Type Mapping
+
+Policy items without `action-matches` are checked by access type only.
+
+| S3 Action | Resource | Access type | Description |
+|:-----------------------------------|:---------|:---------------------|:----------------------------------------------------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` | Read the bucket's Object Lock settings. |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE` | Set the bucket's default retention (mode, period, event hold). |
+| `GetObjectRetention` | Key | `READ` | Read the retention mode, date and event hold. |
+| `PutObjectRetention` | Key | `WRITE` | Set, extend, shorten or remove a retention; set or release an event hold. |
+| `GetObjectLegalHold` | Key | `READ` | Read the legal hold. |
+| `PutObjectLegalHold` | Key | `WRITE` | Set the legal hold `ON` or `OFF`. |
+| `BypassGovernanceRetention` | Key | `WRITE` and `DELETE` | Bypass Governance retention. |
+
+##### `action-matches`
+
+* With `enableActionMatcherInPoliciesCondition`, a policy item can have an `action-matches` condition: a list of S3 actions (`*` and `Prefix*` allowed)
+ that `RangerActionMatcher` compares with the action of the request.
+* It only narrows an item: the item applies when both the access type and the action match. Items without it behave as before.
+* Example: grant a legal team `WRITE` with `action-matches: PutObjectLegalHold`, so it can change legal holds but not retention.
+* S3 Gateway sets the action in `S3GActionIamMapper`, and OM passes it to Ranger in the `RequestContext`. OM only adds the dependent actions below.
+* Today S3 Gateway only sends the action when `ozone.s3g.sts.http.enabled=true`. This design depends on [HDDS-17054](https://issues.apache.org/jira/browse/HDDS-17054), which sends it for every S3 request.
+* For requests without an S3 action (ofs, o3fs, Ozone CLI, Java API), see [Trusted Boundary](#trusted-boundary-and-compliance-scope).
+
+##### Dependent Permissions
+
+As in AWS S3 ([required permissions for S3 API operations](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html)),
+some requests also need a second action, depending on their headers or on the key:
+
+| S3 API | Main S3 action | Header or condition | Dependent S3 action | Access type | If denied |
+|:----------------------------------------------------------|:---------------------|:-------------------------------------------------------------------------------------------------------|:-------------------------------------------|:------------------------|:--------------------------------------------|
+| `CreateBucket` | `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | `PutBucketObjectLockConfiguration` | `WRITE` (bucket) | Request fails |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `PutObject` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | `PutObjectRetention` | `WRITE` (key) | Request fails |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `PutObject` | `x-amz-object-lock-legal-hold` | `PutObjectLegalHold` | `WRITE` (key) | Request fails |
+| `GetObject`, `HeadObject` | `GetObject` | The key has a retention or a legal hold | `GetObjectRetention`, `GetObjectLegalHold` | `READ` (key) | OM drops the lock information, so S3 Gateway omits the `x-amz-object-lock-*` headers; the read succeeds |
+| `DeleteObject`, `DeleteObjects` | `DeleteObject` | `x-amz-bypass-governance-retention: true` on a key under Governance retention | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails (`DeleteObjects`: per-key `AccessDenied`) |
+| `PutObjectRetention` | `PutObjectRetention` | Shortens or removes a Governance retention | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails |
+| `PutObject`, `CopyObject`, `CompleteMultipartUpload`, rename (native clients) | `PutObject` | Overwrites or renames a key under Governance retention with `bypassGovernanceRetention` | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails |
+
+The dependent actions use the same access types as data (`READ` / `WRITE`, and `DELETE` for the bypass), so they mostly matter for policy items with `action-matches`.
+With such items, a data writer cannot set its own retention with a header, and a retention manager cannot upload data.
+
+**How it works**: S3 Gateway sends one RPC with the main action. OM calls `checkAccess` once per action, each with one access type and one action, and every call must pass.
Review Comment:
It is one RPC already. In preExecute, OM runs the normal PutObject check, then adds one checkAccess per Object Lock header or flag in the request, for example (WRITE, PutObjectRetention).
The Ranger plugin evaluates in-process against cached policies, so the extra check is not a round trip. One Ranger evaluation for both is not possible with action-matches, since a request carries one action and the two actions may be allowed by different policy items. Rewrote "How it works" under Dependent Permissions and Alternatives to explain this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-5691440872
@devmadhuu @ChenSammi @sumitagrawl PTAL
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266828574
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
Review Comment:
Let's change this to KeyRetentionConfig or KeyRention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266963929
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
+one by one from the client, so it is blocked by Object Lock as well. Ozone has no account concept, so unlike AWS
+(where closing the account destroys all data, including Compliance-locked objects), there would be no way to decommission
+a tenant or volume until every retention period expires. An admin-only command is provided for this case,
+with the volume as the unit of deletion, as the closest analogue to deleting an AWS account:
+
+```text
+ozone admin om volume force-delete <volume>
+```
+
+* **Disabled by default**: guarded by a new OM config (for example `ozone.om.object.lock.volume.force.delete.enabled`, default `false`).
+ Deployments that must never allow removal of Compliance-locked data keep it disabled.
+* **Authorization**: OM administrators only (`ozone.administrators`). It is not exposed through S3 Gateway and is not covered by `BypassGovernanceRetention`.
+* **Execution**: OM marks the volume as deleting and rejects any new request on it. A background service then purges buckets and keys in batches
+ (including open keys, multipart uploads and snapshots), because a single Ratis transaction cannot delete a large volume. WORM checks are skipped only for this purge.
+ Deletion is irreversible.
+* **Audit**: the request and the purge result are written to the OM audit log.
+* **Trusted boundary**: this is the only way to remove Compliance-locked data through the Ozone API. The data is protected from everyone except a cluster administrator who explicitly enabled and ran this command.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+The additional metadata lookups are expected to have a small impact on throughput and latency; this will be verified with benchmarks and metrics during implementation.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions.
+
+By leveraging the **STS action-matching framework**, Ozone maps standard S3 API actions (e.g., `BypassGovernanceRetention`, `PutObjectLegalHold`) directly to standard Ranger primitive access types (e.g., `WRITE` and `READ`). This fine-grained, action-level authorization enforces the Principle of Least Privilege **without requiring custom dedicated access types**. It effectively prevents unauthorized tampering or removal of locks, ensuring robust compliance and data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+* **System Clock Tampering**: Artificially manipulating or rolling back the system clock (NTP/OS time) on OM or DataNode hosts to prematurely expire object retention periods or evade time-based WORM validation.
+* **Ranger Super Admin Privilege Abuse**: Modifying, disabling, or overriding Ozone authorization policies via Apache Ranger admin credentials (or direct access to Ranger’s backend database), allowing privileged operators to alter bucket WORM flags or grant unrestricted deletion permissions outside normal enforcement workflows.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+* **Table Schema Changes**: Since all newly introduced fields in the Bucket Table (`objectLockEnabled`, `defaultRetention`) and Key Table (`retention`, `legalHold`) are defined as `optional` in Protobuf, they are wire-compatible.
+ To avoid RocksDB serialization problems if the cluster is rolled back after new fields were written, a new OM layout feature gates Object Lock: until the upgrade is finalized, OM rejects every Object Lock request (bucket configuration, retention, legal hold, and `x-amz-object-lock-*` headers on Put / Multipart Upload), so no Object Lock metadata is persisted before finalization.
+* **OM Version Leveling**: OM version leveling is introduced primarily because this feature adds new OM APIs (such as `setRetentionConfig` and `setLegalHold`). The new OM version allows clients to determine whether the target OM cluster supports executing these Object Lock APIs during rolling upgrades.
+
+## Alternatives
+
+### 1. Why Not Use Native ACLs for Object Lock Management?
+
+* **Proposal**:
+ Support Native ACLs alongside or instead of Ranger to configure and manage Object Lock states (e.g., using Ozone native ACLs or POSIX-like ACLs at the Volume/Bucket/Key level).
+
+* **Why Rejected**:
+ 1. **Conflicting Security & Authorization Philosophies**:
+ Native ACLs in Ozone are primarily designed for granular, resource-centric permissions (often granting permissions directly to specific objects/buckets or inherited via path hierarchies). In contrast, Object Lock is an enterprise compliance feature that requires strict, role-based access governance (RBAC/ABAC). Compliance controls like WORM and Legal Holds are inherently tied to compliance officers, security auditors, or legal teams—not individual data owners.
+ 2. **Risk of Decentralized & Inconsistent Governance**:
+ Native ACLs allow object creators or bucket owners to adjust permissions. If lock controls were exposed through Native ACLs, standard bucket administrators or key owners could modify or override lock-related ACLs locally. Centralizing lock authorization exclusively in **Apache Ranger** ensures that compliance policies are managed from a unified pane of glass, preventing local policy drift or accidental misconfigurations.
+ 3. **Operational Complexity**:
+ Supporting dual authorization models (Ranger + Native ACLs) for critical compliance features introduces unnecessary complexity, potential security loopholes, and ambiguous resolution hierarchies during audits. Excluding Native ACLs ensures clean boundary enforcement.
+
+## Plan
+
+The implementation of S3 Object Lock in Apache Ozone is divided into six structured phases. This ensures incremental delivery, clean service boundaries, and testability at each stage.
+
+### Phase 1: Metadata Infrastructure & Schema Definitions
Review Comment:
For phases, I would suggest we only list the highlights, instead of current detail things, unless these are the steps guides for later contributors which are not familiar with the design. The reason is that details things have big chance to change later during the implementation, while the author/review of implementation usually will forget the update/check the design doc, cause this part stale in the doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4357941298
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
Review Comment:
Can we make this requires ozone administrator too? I felt it's kind of complex to remember these detail difference that with or without RetainUntilDate. That's to say, COMPLIANCE mode set/extend requires Ozone administrator
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4278292092
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
Review Comment:
Added a Usage section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279884353
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
+one by one from the client, so it is blocked by Object Lock as well. Ozone has no account concept, so unlike AWS
+(where closing the account destroys all data, including Compliance-locked objects), there would be no way to decommission
+a tenant or volume until every retention period expires. An admin-only command is provided for this case,
+with the volume as the unit of deletion, as the closest analogue to deleting an AWS account:
+
+```text
+ozone admin om volume force-delete <volume>
+```
+
+* **Disabled by default**: guarded by a new OM config (for example `ozone.om.object.lock.volume.force.delete.enabled`, default `false`).
+ Deployments that must never allow removal of Compliance-locked data keep it disabled.
+* **Authorization**: OM administrators only (`ozone.administrators`). It is not exposed through S3 Gateway and is not covered by `BypassGovernanceRetention`.
+* **Execution**: OM marks the volume as deleting and rejects any new request on it. A background service then purges buckets and keys in batches
+ (including open keys, multipart uploads and snapshots), because a single Ratis transaction cannot delete a large volume. WORM checks are skipped only for this purge.
+ Deletion is irreversible.
+* **Audit**: the request and the purge result are written to the OM audit log.
+* **Trusted boundary**: this is the only way to remove Compliance-locked data through the Ozone API. The data is protected from everyone except a cluster administrator who explicitly enabled and ran this command.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+The additional metadata lookups are expected to have a small impact on throughput and latency; this will be verified with benchmarks and metrics during implementation.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions.
+
+By leveraging the **STS action-matching framework**, Ozone maps standard S3 API actions (e.g., `BypassGovernanceRetention`, `PutObjectLegalHold`) directly to standard Ranger primitive access types (e.g., `WRITE` and `READ`). This fine-grained, action-level authorization enforces the Principle of Least Privilege **without requiring custom dedicated access types**. It effectively prevents unauthorized tampering or removal of locks, ensuring robust compliance and data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+* **System Clock Tampering**: Artificially manipulating or rolling back the system clock (NTP/OS time) on OM or DataNode hosts to prematurely expire object retention periods or evade time-based WORM validation.
+* **Ranger Super Admin Privilege Abuse**: Modifying, disabling, or overriding Ozone authorization policies via Apache Ranger admin credentials (or direct access to Ranger’s backend database), allowing privileged operators to alter bucket WORM flags or grant unrestricted deletion permissions outside normal enforcement workflows.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+* **Table Schema Changes**: Since all newly introduced fields in the Bucket Table (`objectLockEnabled`, `defaultRetention`) and Key Table (`retention`, `legalHold`) are defined as `optional` in Protobuf, they are wire-compatible.
+ To avoid RocksDB serialization problems if the cluster is rolled back after new fields were written, a new OM layout feature gates Object Lock: until the upgrade is finalized, OM rejects every Object Lock request (bucket configuration, retention, legal hold, and `x-amz-object-lock-*` headers on Put / Multipart Upload), so no Object Lock metadata is persisted before finalization.
+* **OM Version Leveling**: OM version leveling is introduced primarily because this feature adds new OM APIs (such as `setRetentionConfig` and `setLegalHold`). The new OM version allows clients to determine whether the target OM cluster supports executing these Object Lock APIs during rolling upgrades.
+
+## Alternatives
+
+### 1. Why Not Use Native ACLs for Object Lock Management?
+
+* **Proposal**:
+ Support Native ACLs alongside or instead of Ranger to configure and manage Object Lock states (e.g., using Ozone native ACLs or POSIX-like ACLs at the Volume/Bucket/Key level).
+
+* **Why Rejected**:
+ 1. **Conflicting Security & Authorization Philosophies**:
+ Native ACLs in Ozone are primarily designed for granular, resource-centric permissions (often granting permissions directly to specific objects/buckets or inherited via path hierarchies). In contrast, Object Lock is an enterprise compliance feature that requires strict, role-based access governance (RBAC/ABAC). Compliance controls like WORM and Legal Holds are inherently tied to compliance officers, security auditors, or legal teams—not individual data owners.
+ 2. **Risk of Decentralized & Inconsistent Governance**:
+ Native ACLs allow object creators or bucket owners to adjust permissions. If lock controls were exposed through Native ACLs, standard bucket administrators or key owners could modify or override lock-related ACLs locally. Centralizing lock authorization exclusively in **Apache Ranger** ensures that compliance policies are managed from a unified pane of glass, preventing local policy drift or accidental misconfigurations.
+ 3. **Operational Complexity**:
+ Supporting dual authorization models (Ranger + Native ACLs) for critical compliance features introduces unnecessary complexity, potential security loopholes, and ambiguous resolution hierarchies during audits. Excluding Native ACLs ensures clean boundary enforcement.
+
+## Plan
+
+The implementation of S3 Object Lock in Apache Ozone is divided into six structured phases. This ensures incremental delivery, clean service boundaries, and testability at each stage.
+
+### Phase 1: Metadata Infrastructure & Schema Definitions
+
+*Goal: Establish protobuf schemas, in-memory domain models, and RocksDB table serialization codecs.*
+
+1. **Protobuf Definitions (`OmClientProtocol.proto`)**:
+ - Introduce retention domain messages:
+ - `RetentionMode` enum (`GOVERNANCE = 1`, `COMPLIANCE = 2`).
+ - `Rule` message (`retentionMode`, `timeUnit`, `duration`).
+ - `EventHold` message (`enabled`, `rule`).
+ - `RetentionConfig` message (`retentionMode`, `rule`, `eventHold`).
+ - `Retention` message (`retentionMode`, `retentionDate`, `eventHold`).
+ - Extend `BucketInfo` (OBS buckets only):
+ - `optional bool objectLockEnabled = 24 [default = false];`
+ - `optional RetentionConfig defaultRetention = 25;`
+ - Extend `KeyInfo` & `KeyInfoProtoLight`:
+ - `optional Retention retention = 23;`
+ - `optional bool legalHold = 24 [default = false];`
+
+2. **Domain Models & Helpers (`hadoop-ozone/common`)**:
+ - Update `OmBucketInfo` / `OmBucketInfo.Builder` with getters, setters, and protobuf translation.
+ - Update `OmKeyInfo` / `OmKeyInfo.Builder` with getters, setters, and protobuf translation.
+ - Introduce new class `Retention` to encapsulate bucket rules of retention:
+ - `fromProto` to convert from protobuf to java class.
+ - `toProto` to convert from java class to protobuf.
+ - Introduce `RetentionUtil` to provide utility methods for retention calculations:
+ - Check if `retention` (`retentionDate`, `eventHold`) is still locking: a key is locked if `legalHold` is true, `eventHold.enabled` is true, or `retentionDate` is later than the request time stamped by the leader in `preExecute` (not the local clock of each OM).
+ - Validate retention rules (e.g., duration > 0, valid time unit, etc.).
+
+---
+
+### Phase 2: Ranger Integration & Action Mapping
+
+*Goal: Enable fine-grained access control separating standard data mutating permissions from compliance actions.*
+
+1. **Ranger Action Mapping & UI Validation (`ozone.json`)**:
+ - Do **NOT** create new underlying Ranger access types. Retain primitive access types (`READ`, `WRITE`, `DELETE`) and leverage the STS action-matching framework (`action-matches`).
+ - Register the 7 S3 Object Lock actions in `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json`:
+ - `GetBucketObjectLockConfiguration` (Bucket level: requires `READ`)
+ - `PutBucketObjectLockConfiguration` (Bucket level: requires `WRITE`)
+ - `GetObjectRetention` (Key level: requires `READ`)
+ - `PutObjectRetention` (Key level: requires `WRITE`)
+ - `GetObjectLegalHold` (Key level: requires `READ`)
+ - `PutObjectLegalHold` (Key level: requires `WRITE`)
+ - `BypassGovernanceRetention` (Key level: requires `WRITE` / `DELETE`)
+ - Enable design-time validation in Ranger React UI to ensure administrators cannot grant compliance actions without selecting their required primitive permissions.
+
+2. **Payload-Aware String Matcher Condition Evaluator**:
+ - Register a dedicated custom condition evaluator (e.g., `RangerOzoneValueMatcher`, extending `RangerAbstractConditionEvaluator`) in `ranger-servicedef-ozone.json` under `policyConditions`.
+ - Update `RangerOzoneAuthorizer` and request context handling to extract sensitive payload attributes and inject them into `RangerAccessRequest.getContext()`:
+ - `s3:object-lock-legal-hold`: Extracted from the request body (`ON` vs. `OFF`).
+ - `s3:object-lock-mode`: Extracted from the request body (`GOVERNANCE` vs. `COMPLIANCE`).
+ - `s3:x-amz-bypass-governance-retention`: Extracted from request headers (`true` vs. `false`).
+ - Implement runtime string matching within the evaluator to enforce value-level role segregation:
+ - Allow decoupling privileged operations (e.g., granting `PutObjectLegalHold` exclusively when `s3:object-lock-legal-hold == "ON"`, while restricting `OFF` to compliance officers).
+ - Prevent accidental or unauthorized lock-in by enforcing `s3:object-lock-mode == "GOVERNANCE"` for standard operators, denying `COMPLIANCE` configuration.
+
+3. **S3 Gateway & IAM Action Resolution**:
+ - Update `S3GAction` in `hadoop-ozone/s3gateway`: add audit actions for all lock configuration operations.
+ - Update `S3GActionIamMapper` in `hadoop-ozone/s3gateway`: map the new audit actions to standard IAM S3 actions (`GetBucketObjectLockConfiguration`, `PutBucketObjectLockConfiguration`, `GetObjectRetention`, `PutObjectRetention`, `GetObjectLegalHold`, `PutObjectLegalHold`, `BypassGovernanceRetention`).
+ - Update `IamSessionPolicyResolver.S3Action` in `hadoop-ozone/common`: register the actions with their resource scopes (Bucket vs. Object) and base permissions (`READ`, `WRITE`).
+ - Propagate `s3Action` via `S3Auth` through OM RPC to `RequestContext.s3Action` for evaluation by `RangerOzoneAuthorizer`.
+
+---
+
+### Phase 3: Object Lock Management APIs
+
+*Goal: Implement S3 REST endpoints, OM client RPC protocols, and OM HA consensus handlers for configuring and querying locks.*
+
+#### 1. Bucket Object Lock Configuration
+- **`PutBucketObjectLockConfiguration`**:
+ - *S3G*: Route `PUT /{bucket}?object-lock`. Parse `ObjectLockConfiguration` XML (`ObjectLockEnabled`, optional `DefaultRetention`).
+ - *RPC*: Add `SetBucketObjectLockConfigRequest` / `Response` in `OmClientProtocol.proto`.
+ - *OM Handler (`OMBucketSetObjectLockConfigRequest`)*:
+ - `preExecute`: Validate Ranger action `PutBucketObjectLockConfiguration`. Fail-fast on invalid parameters.
+ - `validateAndUpdateCache`: Under bucket write lock, check bucket exists and layout is OBS. Ensure immutability: if `objectLockEnabled` is already `true`, it cannot be toggled to `false`. Set `objectLockEnabled` and update `defaultRetention`. Commit to `BucketTable`.
Review Comment:
It follows the link: PutBucketObjectLockConfiguration stores the configuration on the source bucket and GetBucketObjectLockConfiguration reads it from there, with authorization checked on both the link and the source bucket. Key operations already resolve links, so WORM checks always use the source bucket's configuration. Added this to the doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279888336
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
Review Comment:
Good catch, native rename on OBS buckets was missing. Added this to the doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ivandika3 (via GitHub)" <gi...@apache.org>.
ivandika3 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4290659618
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
Currently, we have no immediate requirement for WORM support.
I agree with @ChenSammi proposal of using READ and owner and the admin rather than using READ_ACL and WRITE_ACL, but I will leave it up to the community to decide. Our cluster native ACL model is different so even if we decided to backport this in the future, we will need to adapt it to our own ACL model.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4356578818
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
Review Comment:
This conflicts with above "setting or changing a Compliance retention needs an Ozone administrator " statement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4345196339
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
This line says "Ranger has no administrator role", but OM's Ozone administrator check (`ozone.administrators`) doesn't depend on the authorizer. OM already uses it on Ranger clusters, for example `OMSnapshotCreateRequest` only allows an Ozone admin or the bucket owner to create snapshots.
This matters because with default Ranger settings (`action-matches` off), any user with `WRITE` on `bucket1/*` can set `COMPLIANCE` with a date 100 years out on every key in the bucket. Nobody can undo that through the API; the only fix is stopping all OMs and running the offline repair.
Could Compliance changes need an Ozone administrator in both models, the same as with native ACLs?
Just this clarification I need.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4313481283
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,589 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets, with authorization through Ozone native ACLs or Apache Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in Ozone is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design supports both Ozone native ACLs and Apache Ranger to manage Object Lock permissions on OBS buckets.
+Native ACLs are used by deployments without Ranger and allow the core logic to be implemented and tested independently of the Ranger integration.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, with permission and access control through native ACLs or Ranger.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Value-Level Ranger Authorization: a payload-aware Ranger condition evaluator (for example, allowing a legal hold to be set `ON` but not `OFF`) is deferred to a future phase.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Authorization: Ozone has no root account.
+ * With native ACLs, changing Object Lock settings requires `WRITE_ACL` and bypassing Governance retention is limited to Ozone administrators (see [Native ACL Mapping](#1-native-acl-mapping)).
+ * With Ranger, AWS-style action-level authorization for S3 requests requires both `ozone.s3g.sts.http.enabled` and `enableActionMatcherInPoliciesCondition` to be enabled; both are `false` by default (see [Prerequisite for S3 Fine-Grained Authorization](#3-prerequisite-for-s3-fine-grained-authorization)).
+ * Otherwise, and for native clients, Ranger falls back to `READ_ACL` / `WRITE_ACL`, and bypassing Governance retention requires `WRITE_ACL`, `WRITE` and `DELETE` (see [Fallback: Base Permission Mode](#4-fallback-base-permission-mode)).
+ * A Compliance retention cannot be removed through any API; it can only be changed offline with `ozone repair` (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until a principal allowed to change the legal hold (`WRITE_ACL` on the key, plus the `PutObjectLegalHold` action in Ranger Fine-Grained Mode) explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+* **Usage**:
+ * Used to preserve data for litigation, investigations or audits, when it is not known in advance how long the data must be kept.
+ * A legal hold has no mode and no expiration date. It is independent of retention: a key can have a legal hold, a retention period, or both,
+ and it stays locked until the legal hold is removed and the retention period has expired.
+ * A legal hold can be placed on a key in an Object Lock enabled bucket even if the key has no retention.
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * The bucket default can have both `rule` and `eventHold`. In that case a new key gets `retentionDate` = creation time + `rule` and an active event hold;
+ the key stays locked until the event hold is released and `retentionDate` has passed. With only `eventHold`, `retentionDate` is unset until the hold is released.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
Review Comment:
Agreed, the previous design could release an event hold too early. The doc now follows the AWS S3 variable retention API (`EventHold` / `EventHoldDuration`):
- Only an explicit `EventHold=OFF` releases the hold; any other update keeps it active.
- The duration becomes a fixed date only on release (the later of the existing date and release time + duration).
- Changing the duration can only extend the date, never shorten it. Only an explicit earlier `RetainUntilDate` with the Governance bypass can shorten it (never for Compliance).
See "Event Hold Trigger Workflow" and the "Retention Updates" table for the full rules, including the `400` cases. I verified them against AWS S3 with the AWS CLI, since the AWS docs do not cover all of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4311746581
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
@ChenSammi @fmorg-git Updated the doc:
**Native ACLs** (following @ChenSammi's proposal):
- `GetBucketObjectLockConfiguration` / `GetObjectRetention` / `GetObjectLegalHold` require `READ`.
- Changing a legal hold, a Governance retention, or the bucket configuration is limited to the owner of the volume or bucket (the same owner the native authorizer already lets skip ACL checks), or an Ozone administrator. The owner can also bypass Governance retention.
- Setting a Compliance retention is limited to Ozone administrators: setting it on a key (`PutObjectRetention` or the `x-amz-object-lock-*` headers), changing Governance to Compliance, and setting Compliance as a bucket default. Keys that inherit a Compliance bucket default do not need an extra check. Extending a Compliance retention or releasing its event hold only needs the owner, as in AWS S3 where it only needs `PutObjectRetention`.
- @fmorg-git, this also answers the `READ_ACL` question: native ACLs no longer use `READ_ACL` / `WRITE_ACL` for Object Lock. Native ACLs have no actions, so instead of approximating them with ACL types, they use ownership and the administrator role, which is stricter.
**Ranger**:
- The doc now describes two authorization models, native ACLs and Ranger, and recommends enabling `enableActionMatcherInPoliciesCondition` for fine-grained permissions. I removed the STS flag combinations and the "fallback" wording.
- The doc depends on HDDS-17054, so S3 Gateway always sends the S3 action.
- @fmorg-git, on the concern about actions being set in several places: the new OM requests do not set S3 actions.
- With Ranger, OM maps each operation to an access type, and `action-matches` only narrows a policy item. Requests without an S3 action (ofs, o3fs, CLI, native Java API) are matched only by items without the condition or with `*`, so switching to a non-S3 client cannot get around an action restriction. This is now described in the Trusted Boundary section; deployments with both S3 and non-S3 access should keep such items for non-S3 clients.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4398944370
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
Review Comment:
I'd keep WRITE for lock changes. There's no "remove lock" operation; modifying Governance uses PutObjectRetention with BypassGovernanceRetention (WRITE and DELETE), while non-governance changes only need WRITE. Forcing DELETE for all retention calls would break WORM roles ("can lock, never delete"). This matches AWS, where only BypassGovernanceRetention is separate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4415345132
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Setting Compliance only needs `PutObjectRetention` (`WRITE`), as in AWS S3, so it can be delegated per bucket with Ranger policies.
+ Requiring an Ozone administrator, as with native ACLs, would make it cluster wide.
+ Only `action-matches` can limit who sets it. No Ranger permission can remove a Compliance retention; only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+##### Access Type Mapping
+
+Policy items without `action-matches` are checked by access type only.
+
+| S3 Action | Resource | Access type | Description |
+|:-----------------------------------|:---------|:---------------------|:----------------------------------------------------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` | Read the bucket's Object Lock settings. |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE` | Set the bucket's default retention (mode, period, event hold). |
+| `GetObjectRetention` | Key | `READ` | Read the retention mode, date and event hold. |
+| `PutObjectRetention` | Key | `WRITE` | Set, extend, shorten or remove a retention; set or release an event hold. |
+| `GetObjectLegalHold` | Key | `READ` | Read the legal hold. |
+| `PutObjectLegalHold` | Key | `WRITE` | Set the legal hold `ON` or `OFF`. |
+| `BypassGovernanceRetention` | Key | `WRITE` and `DELETE` | Bypass Governance retention. |
+
+##### `action-matches`
+
+* With `enableActionMatcherInPoliciesCondition`, a policy item can have an `action-matches` condition: a list of S3 actions (`*` and `Prefix*` allowed)
+ that `RangerActionMatcher` compares with the action of the request.
+* It only narrows an item: the item applies when both the access type and the action match. Items without it behave as before.
+* Example: grant a legal team `WRITE` with `action-matches: PutObjectLegalHold`, so it can change legal holds but not retention.
+* S3 Gateway sets the action in `S3GActionIamMapper`, and OM passes it to Ranger in the `RequestContext`. OM only adds the dependent actions below.
+* Today S3 Gateway only sends the action when `ozone.s3g.sts.http.enabled=true`. This design depends on [HDDS-17054](https://issues.apache.org/jira/browse/HDDS-17054), which sends it for every S3 request.
+* For requests without an S3 action (ofs, o3fs, Ozone CLI, Java API), see [Trusted Boundary](#trusted-boundary-and-compliance-scope).
+
+##### Dependent Permissions
+
+As in AWS S3 ([required permissions for S3 API operations](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-with-s3-policy-actions.html)),
+some requests also need a second action, depending on their headers or on the key:
+
+| S3 API | Main S3 action | Header or condition | Dependent S3 action | Access type | If denied |
+|:----------------------------------------------------------|:---------------------|:-------------------------------------------------------------------------------------------------------|:-------------------------------------------|:------------------------|:--------------------------------------------|
+| `CreateBucket` | `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | `PutBucketObjectLockConfiguration` | `WRITE` (bucket) | Request fails |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `PutObject` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | `PutObjectRetention` | `WRITE` (key) | Request fails |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `PutObject` | `x-amz-object-lock-legal-hold` | `PutObjectLegalHold` | `WRITE` (key) | Request fails |
+| `GetObject`, `HeadObject` | `GetObject` | The key has a retention or a legal hold | `GetObjectRetention`, `GetObjectLegalHold` | `READ` (key) | OM drops the lock information, so S3 Gateway omits the `x-amz-object-lock-*` headers; the read succeeds |
+| `DeleteObject`, `DeleteObjects` | `DeleteObject` | `x-amz-bypass-governance-retention: true` on a key under Governance retention | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails (`DeleteObjects`: per-key `AccessDenied`) |
+| `PutObjectRetention` | `PutObjectRetention` | Shortens or removes a Governance retention | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails |
+| `PutObject`, `CopyObject`, `CompleteMultipartUpload`, rename (native clients) | `PutObject` | Overwrites or renames a key under Governance retention with `bypassGovernanceRetention` | `BypassGovernanceRetention` | `WRITE`, `DELETE` (key) | Request fails |
+
+The dependent actions use the same access types as data (`READ` / `WRITE`, and `DELETE` for the bypass), so they mostly matter for policy items with `action-matches`.
+With such items, a data writer cannot set its own retention with a header, and a retention manager cannot upload data.
+
+**How it works**: S3 Gateway sends one RPC with the main action. OM calls `checkAccess` once per action, each with one access type and one action, and every call must pass.
Review Comment:
It is one RPC already. In preExecute, OM runs the normal PutObject check, then adds one checkAccess per Object Lock header or flag in the request, for example (WRITE, PutObjectRetention).
The Ranger plugin evaluates in-process against cached policies, so the extra check is not a round trip. One Ranger evaluation for both is not possible with action-matches, since a request carries one action and the two actions may be allowed by different policy items. Rewrote "How it works" under Dependent Permissions and Alternatives #2 to explain this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4036302250
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,559 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required RetentionMode retentionMode = 1;
+ required TimeUnit timeUnit = 2;
+ required uint64 duration = 3;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+* **retentionDate**: Represents the expiration timestamp of the retention lock. While clients or API calls specify retention duration in terms of days or years, Ozone calculates the definitive expiration date upon applying the rule and persists it as a timestamp into key table.
+* **legalHold**: Indicates whether an explicit legal hold is active on the key.
+
+Two new fields: retentionDate & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional string retentionDate = 23;
+ optional RetentionConfig retentionConfig = 24;
+ optional bool legalHold = 25 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+| :--- | :--- | :--- | :--- |
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` / `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly specify the `x-amz-bypass-governance-retention: true` header in the delete or overwrite request.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**:
+> _Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them due to unsupporting Versioning, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+Preliminary assessments indicate that the additional metadata lookups impose an acceptable impact on overall system throughput and latency. Metrics will be continuously monitored and tuned as necessary.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions. Dedicated Access Types (BYPASS_GOVERNANCE, PUT_LEGAL_HOLD, etc.) enforce the Principle of Least Privilege, preventing unauthorized tampering or removal of locks and enhancing data immutability.
Review Comment:
Good catch! I've updated the description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279040287
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
Review Comment:
Agreed. Add a section for Offline Compliance Retention Repair.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279021239
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
Review Comment:
Agreed. Removed the value-level Ranger condition evaluator from this phase and moved it to Non-Goal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4313481283
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,589 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets, with authorization through Ozone native ACLs or Apache Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in Ozone is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design supports both Ozone native ACLs and Apache Ranger to manage Object Lock permissions on OBS buckets.
+Native ACLs are used by deployments without Ranger and allow the core logic to be implemented and tested independently of the Ranger integration.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, with permission and access control through native ACLs or Ranger.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Value-Level Ranger Authorization: a payload-aware Ranger condition evaluator (for example, allowing a legal hold to be set `ON` but not `OFF`) is deferred to a future phase.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Authorization: Ozone has no root account.
+ * With native ACLs, changing Object Lock settings requires `WRITE_ACL` and bypassing Governance retention is limited to Ozone administrators (see [Native ACL Mapping](#1-native-acl-mapping)).
+ * With Ranger, AWS-style action-level authorization for S3 requests requires both `ozone.s3g.sts.http.enabled` and `enableActionMatcherInPoliciesCondition` to be enabled; both are `false` by default (see [Prerequisite for S3 Fine-Grained Authorization](#3-prerequisite-for-s3-fine-grained-authorization)).
+ * Otherwise, and for native clients, Ranger falls back to `READ_ACL` / `WRITE_ACL`, and bypassing Governance retention requires `WRITE_ACL`, `WRITE` and `DELETE` (see [Fallback: Base Permission Mode](#4-fallback-base-permission-mode)).
+ * A Compliance retention cannot be removed through any API; it can only be changed offline with `ozone repair` (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until a principal allowed to change the legal hold (`WRITE_ACL` on the key, plus the `PutObjectLegalHold` action in Ranger Fine-Grained Mode) explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+* **Usage**:
+ * Used to preserve data for litigation, investigations or audits, when it is not known in advance how long the data must be kept.
+ * A legal hold has no mode and no expiration date. It is independent of retention: a key can have a legal hold, a retention period, or both,
+ and it stays locked until the legal hold is removed and the retention period has expired.
+ * A legal hold can be placed on a key in an Object Lock enabled bucket even if the key has no retention.
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * The bucket default can have both `rule` and `eventHold`. In that case a new key gets `retentionDate` = creation time + `rule` and an active event hold;
+ the key stays locked until the event hold is released and `retentionDate` has passed. With only `eventHold`, `retentionDate` is unset until the hold is released.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
Review Comment:
Agreed, the previous design could release an event hold too early. The doc now follows the AWS S3 variable retention API:
- `PutObjectRetention` carries `EventHold` (`ON` / `OFF`) and `EventHoldDuration`; `PutObject` / `CreateMultipartUpload` accept the `x-amz-object-lock-event-hold*` headers, and the bucket default can include `DefaultEventHold`.
- Only an explicit `EventHold=OFF` on a key whose hold is `ON` releases it. Any other `PutObjectRetention` update keeps the event hold active.
- The duration becomes a fixed date only on release: OM sets `retentionDate` to the later of the existing date and release time + duration. While the hold is `ON`, `GetObjectRetention` / `HeadObject` return the computed date (the later of now + duration and the stored date).
- Changing the duration never shortens the protection: before a decrease, OM saves the current effective date as the minimum, with or without the bypass.
- Shortening is done only with an earlier `RetainUntilDate` and the Governance bypass (never for Compliance). While the hold is `ON`, that request must also carry `EventHold`, and with `EventHold=ON` the date cannot be earlier than now + duration.
- Invalid requests are rejected with `400`, as in AWS S3: no mode, neither `RetainUntilDate` nor `EventHold`, a duration without `EventHold=ON`, `ON` without exactly one duration, `OFF` with a duration, or `OFF` when there is no future retain-until date.
- A new "Retention Updates" table lists the rules: a longer retention, a shorter event hold duration or releasing the event hold only needs `PutObjectRetention` in both modes; an earlier `RetainUntilDate` needs the Governance bypass and is not allowed in Compliance.
I checked these rules against AWS S3 (Governance mode) with the AWS CLI, since the AWS docs do not cover all of them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4313463230
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
Review Comment:
@ChenSammi @fmorg-git Updated the doc:
**Native ACLs** (following @ChenSammi's proposal):
- `GetBucketObjectLockConfiguration` / `GetObjectRetention` / `GetObjectLegalHold` require `READ`.
- Changing a legal hold, a Governance retention, or the bucket configuration is limited to the owner of the volume or bucket (the same owner the native authorizer already lets skip ACL checks), or an Ozone administrator. The owner can also bypass Governance retention.
- Setting a Compliance retention is limited to Ozone administrators: setting it on a key (`PutObjectRetention` or the `x-amz-object-lock-*` headers), changing Governance to Compliance, and setting Compliance as a bucket default. Keys that inherit a Compliance bucket default do not need an extra check. Extending a Compliance retention or releasing its event hold only needs the owner, as in AWS S3 where it only needs `PutObjectRetention`.
**Ranger**:
- The doc now describes two authorization models, native ACLs and Ranger, and recommends enabling `enableActionMatcherInPoliciesCondition` for fine-grained permissions. I removed the STS flag combinations and the "fallback" wording.
- The doc depends on HDDS-17054, so S3 Gateway always sends the S3 action.
- @fmorg-git, on the concern about actions being set in several places: the new OM requests do not set S3 actions.
- With Ranger, OM maps each operation to an access type, and `action-matches` only narrows a policy item. Requests without an S3 action (ofs, o3fs, CLI, native Java API) are matched only by items without the condition or with `*`, so switching to a non-S3 client cannot get around an action restriction. This is now described in the Trusted Boundary section; deployments with both S3 and non-S3 access should keep such items for non-S3 clients.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-6161351501
@chungen0126 , thanks for updating the design. I have left a few inline comments.
Besides, I would suggest support native ACL too, due to
- some community partners(such as Shopee) use native ACL only
- native ACL support is simple and straightforward
- native ACL can be used to test the major logic easily with Ranger, so major logic testing and Ranger implementation can be proceeded in parallel.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3963561653
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
Review Comment:
I think this question might be for me - I added comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "F64116045 (via GitHub)" <gi...@apache.org>.
F64116045 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4001480104
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,470 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ optional RetentionMode retentionMode = 1;
+ optional uint64 days = 2;
+ optional uint64 years = 3;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted standard Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Action Authorization**: Following the Ranger action-matching model introduced via the [STS](ozone-sts.md), Object Lock actions map directly to AWS S3 action names (without the `s3:` prefix). Access control separates underlying Ranger permissions (e.g., `READ`, `WRITE`) from fine-grained compliance actions:
+ * `GetBucketObjectLockConfiguration`
+ * `PutBucketObjectLockConfiguration`
+ * `GetObjectRetention`
+ * `PutObjectRetention`
+ * `GetObjectLegalHold`
+ * `PutObjectLegalHold`
+ * `BypassGovernanceRetention`
+
+#### 1. Legal Hold Access Control
+
+Authorization is decoupled from the payload value being set, aligning strictly with AWS S3 specifications:
+
+* **`GetObjectLegalHold`**: Allows querying the current Legal Hold status of an object.
+* **`PutObjectLegalHold`**: Governs both setting and clearing Legal Hold status (status ON or OFF). Users must be granted the `PutObjectLegalHold` action (along with `WRITE` permission) in Ranger to toggle the hold.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BypassGovernanceRetention`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Evaluated at the **Key level** via the **`BypassGovernanceRetention`** action.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BypassGovernanceRetention` entitlement in Ranger for that key. Without this permission, even administrators using standard client tools cannot modify or remove the locked object.
+
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
Review Comment:
Thanks @chungen0126 for the detailed design doc!
Following up on that [bucket-level retention is represented by days and years](https://github.com/apache/ozone/pull/11053#discussion_r3954617490), I was wondering how object-level retention is persisted.
Here `PutObjectRetention` specifies an exact `RetainUntilDate`, while `KeyInfo` stores:
```protobuf
optional RetentionConfig retentionConfig = 23;
```
and `RetentionConfig` currently represents retention using `days` and `years`.
Is `RetainUntilDate` derived from these fields, or is it stored somewhere else?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4041692451
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,559 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+, Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional Rule rule = 1;
+ optional EventHold eventHold = 2;
+
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required RetentionMode retentionMode = 1;
+ required TimeUnit timeUnit = 2;
+ required uint64 duration = 3;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+* **retentionDate**: Represents the expiration timestamp of the retention lock. While clients or API calls specify retention duration in terms of days or years, Ozone calculates the definitive expiration date upon applying the rule and persists it as a timestamp into key table.
+* **legalHold**: Indicates whether an explicit legal hold is active on the key.
+
+Two new fields: retentionDate & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional string retentionDate = 23;
+ optional RetentionConfig retentionConfig = 24;
+ optional bool legalHold = 25 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+| :--- | :--- | :--- | :--- |
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` / `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
Review Comment:
thanks for the references @chungen0126 - wasn't aware of it. I implemented Object Lock in the past and hundreds of thousands of customers never needed/asked for it, so I was not aware (also I noticed the AWS article was posted after the implementation). My gut feeling is the complexity required might be overkill compared to how many people might use it (if any). I also note that we currently don't support `Deny` in STS session policy nor any condition key other than `s3:prefix`. In any case, I will defer to @ChenSammi to for her thoughts if it is needed for Ozone.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-5785984461
## Apache Ranger coordination (required)
Object Lock v1 is **Ranger-centric** for lock administration and governance bypass. This is **not** a new Ranger Admin REST API, but it **does require Ranger release / servicedef work** in addition to Ozone changes. Worth calling out explicitly in the OEP (Compatibility or Plan) so operators know minimum Ranger version and cross-repo ordering.
**Required in Apache Ranger (typical paths):**
1. **`security-admin/.../actionRequirements/ozone.json`** — Register the 7 S3 Object Lock actions (`GetBucketObjectLockConfiguration`, `PutBucketObjectLockConfiguration`, `GetObjectRetention`, `PutObjectRetention`, `GetObjectLegalHold`, `PutObjectLegalHold`, `BypassGovernanceRetention`) with required primitive permissions (READ/WRITE/DELETE) at bucket vs key scope. Enables UI validation when authoring policies.
2. **STS / action-matcher prerequisite** — Policies rely on existing **`action-matches`** support (see [ozone-sts.md](ozone-sts.md)), enabled in Ozone via `ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true`. Object Lock does **not** add new Ranger access types; it constrains READ/WRITE/DELETE by S3 action name.
3. **Ozone cluster config** — Operators must enable the action matcher condition before Object Lock policies behave as designed.
**Optional but described in Phase 2 (still Ranger plugin/servicedef):**
4. **`ranger-servicedef-ozone.json`** — Register a policy condition (e.g. `RangerOzoneValueMatcher`) for value-level rules on `s3:object-lock-legal-hold`, `s3:object-lock-mode`, `s3:x-amz-bypass-governance-retention`.
5. **Ranger ozone plugin Java** — Implement the condition evaluator; ensure **`RangerOzoneAuthorizer`** receives S3G-injected context on `RangerAccessRequest`.
**Ozone-side (for completeness):** `S3GActionIamMapper`, `IamSessionPolicyResolver`, propagate `RequestContext.s3Action` (and bypass header) to OM for dual-gate checks.
**Non-Ranger:** Native authorizer is out of scope for lock admin in this OEP — clusters without Ranger need an explicit gap statement.
Suggest adding a short **“Dependencies / Ranger release”** subsection with Jira or release tracking link once Ranger work is filed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "jojochuang (via GitHub)" <gi...@apache.org>.
jojochuang commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-5786434078
### Observability / displaying WORM state
The OEP covers **querying** lock configuration through the standard S3 Object Lock **Get** APIs (`GetBucketObjectLockConfiguration`, `GetObjectRetention`, `GetObjectLegalHold`) and Phase 3 OM / metadata-reader paths. That matches how AWS expects clients to read **bucket default config** and **per-object retention / legal hold** when they call those sub-resources explicitly (AWS CLI, Boto3, etc.).
For **seeing lock state on normal object reads**, AWS documents **`HeadObject`** and **`GetObject`** response headers (when the caller has **`s3:GetObjectRetention`** / **`s3:GetObjectLegalHold`**):
- `x-amz-object-lock-mode`
- `x-amz-object-lock-retain-until-date`
- `x-amz-object-lock-legal-hold`
- (and event-hold related headers where variable retention applies)
The OEP does **not** yet call out enriching **Head/Get** with those headers. That is the main **S3 parity** gap for “display WORM state without extra GET `?retention` / `?legal-hold` round trips.”
**Operator tooling:** The doc does not discuss **`ozone sh` / admin / Recon** (or similar) showing `objectLockEnabled`, default retention, or effective protection on a key. The Java **`ObjectStore`** snippet shows setters only; read paths outside the six S3 Get handlers are not spelled out.
Suggest a short **“Observability”** subsection:
- **v1 minimum (AWS-aligned):** the three Get APIs + document **HeadObject/GetObject** lock headers and required Ranger **`Get*`** actions.
- **Follow-ups (product):** CLI/Recon and read APIs on `ObjectStore` if non-S3 clients need lock metadata.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4356565617
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
Review Comment:
Need the volume or bucket owner -> need the volume owner, bucket owner, or object owner.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4349812257
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
You're right, OM's administrator check does not depend on the authorizer, so "Ranger has no administrator role" was not the right reason. I'd still keep Compliance under Ranger policies instead of requiring an Ozone administrator:
1. A Compliance retention set by mistake is not stuck forever: `ozone repair om object-lock` can remove or downgrade it offline.
2. `ozone.administrators` is cluster wide, so anyone who manages Compliance for a single bucket would have to be an administrator of the whole cluster.
3. It would remove Ranger's fine-grained delegation, for example granting `PutObjectRetention` with `action-matches` to a compliance team on specific buckets only, which is the main reason to use Ranger for this feature.
The risk you describe (action matcher off by default, so any `WRITE` can set Compliance) is called out in the Ranger "Important" note and the Trusted Boundary section, and the Phase 2 user documentation will cover it. The fix is to enable the action matcher and use `action-matches` on `WRITE` items.
Updated the wording of that line to give these reasons instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "devmadhuu (via GitHub)" <gi...@apache.org>.
devmadhuu commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-5632925053
@chungen0126 , just want to highlight to be in compliance with AWS S3 WORM, that AWS has recently added variable retention with event holds to S3 Object Lock, so may be we can think about adding EVENT_TRIGGERED_RETENTION in our existing enum for extended support. Enterprise S3 semantics are increasingly about governance workflows, not just object durability. Reference : [AWS S3 doc](https://aws.amazon.com/about-aws/whats-new/2026/09/amazon-s3-object-lock-variable-retention/)
```
object created
|
event hold
|
business event occurs
|
hold released
|
retention timer starts
|
WORM until timer expires
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4278282353
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
Review Comment:
Done. Head Object and Get/Put Object Tagging are listed as allowed operations for both legal hold and retention.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4398836581
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
Review Comment:
Done. Every PutObjectRetention on a Compliance key now needs an Ozone administrator with native ACLs
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-6504006924
Thanks @ChenSammi for the review. I updated the design doc. Please take a look if you are available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "sumitagrawl (via GitHub)" <gi...@apache.org>.
sumitagrawl commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3843623258
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
Review Comment:
If we introduce this feature in Ozone for validation via above api, Having same at ranger do have any specific advantage ?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+
+### Impact on Write and Delete Flows
+#### Impact on Put Object Flow
+
+Put Flow:
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+#### Impact on Multipart Upload Flow
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. During Initiate Multipart Upload and Put Part phases, WORM checks are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. During the Complete Multipart Upload phase, final validation occurs in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+#### Impact on Delete Object Flow
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
Review Comment:
do impact for bucket removal or volume removal or directory removal where as per soft delete, it moves to deleted directory table, but generally validation is not done to deep child ?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
Review Comment:
Since this object Locking is from S3, what interface via s3 api will be used to configure or manage? or its directly via ozone cli?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
Review Comment:
how ranger and ozone role will combine in validation?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+
+### Impact on Write and Delete Flows
+#### Impact on Put Object Flow
+
+Put Flow:
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+#### Impact on Multipart Upload Flow
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. During Initiate Multipart Upload and Put Part phases, WORM checks are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. During the Complete Multipart Upload phase, final validation occurs in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+#### Impact on Delete Object Flow
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+Preliminary assessments indicate that the additional metadata lookups impose an acceptable impact on overall system throughput and latency. Metrics will be continuously monitored and tuned as necessary.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions. Dedicated Access Types (BYPASS_GOVERNANCE, PUT_LEGAL_HOLD, etc.) enforce the Principle of Least Privilege, preventing unauthorized tampering or removal of locks and enhancing data immutability.
Review Comment:
Like ranger can clear / update policy, and super admin role to update / delete bucket or file policy ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3955707684
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
Review Comment:
@fapifta , could you share some knowledge with @chungen0126 about how to expose new permissions for Ranger, and what are the major tasks needed for Ranger integration, including UI changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3986358284
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
Review Comment:
Object locking can be configured and managed via both the Ozone API and the S3 API.
- Ozone API: It uses OzoneBucket#setRetentionConfig. If no default retention is set, the rule field within retention will simply be null.
- S3 API: It is managed through PutBucketObjectLockConfiguration, which basically follows the standard S3 specifications. You can refer to the official [AWS S3 documentation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLockConfiguration.html) for more details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4290230022
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
@chungen0126 , since we don't have these fine grained permissions in native ACL than Ranger, so instead of using READ_ACL or WRITE_ACL, I would like to propose,
for object and bucket
- READ permission for lock information receive
- owner for lock information update(except the compliance update action)
for compliance mode update
- Only ozone admin
And fallback is not supported, if Ranger is configured, when always use Ranger for the authorization; If Native ACL is configured, when always use Native ACL.
So if user wants the fine grained permission control, then he/she use Ranger, which is our recommendation too for this feature. If Ranger is not available, then Native ACL can be used, which provides the same(or even stricter) level of security protection.
@symious @xichen01 @ivandika3 , will this meet your requirements if in future you guys want to use this feature?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4290230022
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
@chungen0126 , since we don't have these fine grained permissions in native ACL than Ranger, so instead of using READ_ACL or WRITE_ACL, I would like to propose,
for object and bucket
- READ permission for lock information receive
- owner for lock information update(except the compliance update action)
for compliance mode update
- Only ozone admin
And fallback is not supported, if Ranger is configured, when always use Ranger for the authorization; If Native ACL is configured, when always use Native ACL.
So if user wants the fine grained permission control, then he/she use Ranger, which is our recommendation. If Ranger is not available, then Native ACL can be used, which provides the same(or even stricter) level of security protection.
@symious @xichen01 @ivandika3 , will this meet your requirements if in future you guys want to use this feature?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279842408
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
Review Comment:
Updated. The lifecycle service skips locked keys when it selects expired keys, so they are not resent on every run. They are evaluated again in later scans and expire normally once unlocked. The WORM check in DeleteKeys / RenameKey remains the authoritative guard if a key is locked after it was selected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-6186309487
Thanks @fmorg-git @ChenSammi @Gargi-jais11 for the review. Update docs, please take a look if you are available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279744747
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
Review Comment:
Confirmed that s3Action is only forwarded when ozone.s3g.sts.http.enabled=true. The doc now requires both the STS flag and the Ranger action-matcher flag for action-level checks on S3 Gateway requests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-6186372587
I've added the different combinations of `ozone.s3g.sts.http.enabled` and `ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition` to the design doc, though covering all these edge cases makes it quite complex.
If everyone agrees, I would prefer to only document the scenario where **both flags are enabled** for S3 fine-grained access control, and treat all other cases as opting out of S3 action-level permissions (defaulting to base ACL/Ranger permissions). Let me know if that sounds good!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279875687
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
+one by one from the client, so it is blocked by Object Lock as well. Ozone has no account concept, so unlike AWS
+(where closing the account destroys all data, including Compliance-locked objects), there would be no way to decommission
+a tenant or volume until every retention period expires. An admin-only command is provided for this case,
+with the volume as the unit of deletion, as the closest analogue to deleting an AWS account:
+
+```text
+ozone admin om volume force-delete <volume>
+```
+
+* **Disabled by default**: guarded by a new OM config (for example `ozone.om.object.lock.volume.force.delete.enabled`, default `false`).
+ Deployments that must never allow removal of Compliance-locked data keep it disabled.
+* **Authorization**: OM administrators only (`ozone.administrators`). It is not exposed through S3 Gateway and is not covered by `BypassGovernanceRetention`.
+* **Execution**: OM marks the volume as deleting and rejects any new request on it. A background service then purges buckets and keys in batches
+ (including open keys, multipart uploads and snapshots), because a single Ratis transaction cannot delete a large volume. WORM checks are skipped only for this purge.
+ Deletion is irreversible.
+* **Audit**: the request and the purge result are written to the OM audit log.
+* **Trusted boundary**: this is the only way to remove Compliance-locked data through the Ozone API. The data is protected from everyone except a cluster administrator who explicitly enabled and ran this command.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+The additional metadata lookups are expected to have a small impact on throughput and latency; this will be verified with benchmarks and metrics during implementation.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions.
+
+By leveraging the **STS action-matching framework**, Ozone maps standard S3 API actions (e.g., `BypassGovernanceRetention`, `PutObjectLegalHold`) directly to standard Ranger primitive access types (e.g., `WRITE` and `READ`). This fine-grained, action-level authorization enforces the Principle of Least Privilege **without requiring custom dedicated access types**. It effectively prevents unauthorized tampering or removal of locks, ensuring robust compliance and data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+* **System Clock Tampering**: Artificially manipulating or rolling back the system clock (NTP/OS time) on OM or DataNode hosts to prematurely expire object retention periods or evade time-based WORM validation.
+* **Ranger Super Admin Privilege Abuse**: Modifying, disabling, or overriding Ozone authorization policies via Apache Ranger admin credentials (or direct access to Ranger’s backend database), allowing privileged operators to alter bucket WORM flags or grant unrestricted deletion permissions outside normal enforcement workflows.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+* **Table Schema Changes**: Since all newly introduced fields in the Bucket Table (`objectLockEnabled`, `defaultRetention`) and Key Table (`retention`, `legalHold`) are defined as `optional` in Protobuf, they are wire-compatible.
+ To avoid RocksDB serialization problems if the cluster is rolled back after new fields were written, a new OM layout feature gates Object Lock: until the upgrade is finalized, OM rejects every Object Lock request (bucket configuration, retention, legal hold, and `x-amz-object-lock-*` headers on Put / Multipart Upload), so no Object Lock metadata is persisted before finalization.
+* **OM Version Leveling**: OM version leveling is introduced primarily because this feature adds new OM APIs (such as `setRetentionConfig` and `setLegalHold`). The new OM version allows clients to determine whether the target OM cluster supports executing these Object Lock APIs during rolling upgrades.
+
+## Alternatives
+
+### 1. Why Not Use Native ACLs for Object Lock Management?
+
+* **Proposal**:
+ Support Native ACLs alongside or instead of Ranger to configure and manage Object Lock states (e.g., using Ozone native ACLs or POSIX-like ACLs at the Volume/Bucket/Key level).
+
+* **Why Rejected**:
+ 1. **Conflicting Security & Authorization Philosophies**:
+ Native ACLs in Ozone are primarily designed for granular, resource-centric permissions (often granting permissions directly to specific objects/buckets or inherited via path hierarchies). In contrast, Object Lock is an enterprise compliance feature that requires strict, role-based access governance (RBAC/ABAC). Compliance controls like WORM and Legal Holds are inherently tied to compliance officers, security auditors, or legal teams—not individual data owners.
+ 2. **Risk of Decentralized & Inconsistent Governance**:
+ Native ACLs allow object creators or bucket owners to adjust permissions. If lock controls were exposed through Native ACLs, standard bucket administrators or key owners could modify or override lock-related ACLs locally. Centralizing lock authorization exclusively in **Apache Ranger** ensures that compliance policies are managed from a unified pane of glass, preventing local policy drift or accidental misconfigurations.
+ 3. **Operational Complexity**:
+ Supporting dual authorization models (Ranger + Native ACLs) for critical compliance features introduces unnecessary complexity, potential security loopholes, and ambiguous resolution hierarchies during audits. Excluding Native ACLs ensures clean boundary enforcement.
+
+## Plan
+
+The implementation of S3 Object Lock in Apache Ozone is divided into six structured phases. This ensures incremental delivery, clean service boundaries, and testability at each stage.
+
+### Phase 1: Metadata Infrastructure & Schema Definitions
+
+*Goal: Establish protobuf schemas, in-memory domain models, and RocksDB table serialization codecs.*
+
+1. **Protobuf Definitions (`OmClientProtocol.proto`)**:
+ - Introduce retention domain messages:
+ - `RetentionMode` enum (`GOVERNANCE = 1`, `COMPLIANCE = 2`).
+ - `Rule` message (`retentionMode`, `timeUnit`, `duration`).
+ - `EventHold` message (`enabled`, `rule`).
+ - `RetentionConfig` message (`retentionMode`, `rule`, `eventHold`).
+ - `Retention` message (`retentionMode`, `retentionDate`, `eventHold`).
+ - Extend `BucketInfo` (OBS buckets only):
+ - `optional bool objectLockEnabled = 24 [default = false];`
+ - `optional RetentionConfig defaultRetention = 25;`
+ - Extend `KeyInfo` & `KeyInfoProtoLight`:
+ - `optional Retention retention = 23;`
+ - `optional bool legalHold = 24 [default = false];`
+
+2. **Domain Models & Helpers (`hadoop-ozone/common`)**:
+ - Update `OmBucketInfo` / `OmBucketInfo.Builder` with getters, setters, and protobuf translation.
+ - Update `OmKeyInfo` / `OmKeyInfo.Builder` with getters, setters, and protobuf translation.
+ - Introduce new class `Retention` to encapsulate bucket rules of retention:
+ - `fromProto` to convert from protobuf to java class.
+ - `toProto` to convert from java class to protobuf.
+ - Introduce `RetentionUtil` to provide utility methods for retention calculations:
+ - Check if `retention` (`retentionDate`, `eventHold`) is still locking: a key is locked if `legalHold` is true, `eventHold.enabled` is true, or `retentionDate` is later than the request time stamped by the leader in `preExecute` (not the local clock of each OM).
+ - Validate retention rules (e.g., duration > 0, valid time unit, etc.).
+
+---
+
+### Phase 2: Ranger Integration & Action Mapping
+
+*Goal: Enable fine-grained access control separating standard data mutating permissions from compliance actions.*
+
+1. **Ranger Action Mapping & UI Validation (`ozone.json`)**:
+ - Do **NOT** create new underlying Ranger access types. Retain primitive access types (`READ`, `WRITE`, `DELETE`) and leverage the STS action-matching framework (`action-matches`).
+ - Register the 7 S3 Object Lock actions in `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json`:
+ - `GetBucketObjectLockConfiguration` (Bucket level: requires `READ`)
+ - `PutBucketObjectLockConfiguration` (Bucket level: requires `WRITE`)
+ - `GetObjectRetention` (Key level: requires `READ`)
+ - `PutObjectRetention` (Key level: requires `WRITE`)
+ - `GetObjectLegalHold` (Key level: requires `READ`)
+ - `PutObjectLegalHold` (Key level: requires `WRITE`)
+ - `BypassGovernanceRetention` (Key level: requires `WRITE` / `DELETE`)
+ - Enable design-time validation in Ranger React UI to ensure administrators cannot grant compliance actions without selecting their required primitive permissions.
+
+2. **Payload-Aware String Matcher Condition Evaluator**:
+ - Register a dedicated custom condition evaluator (e.g., `RangerOzoneValueMatcher`, extending `RangerAbstractConditionEvaluator`) in `ranger-servicedef-ozone.json` under `policyConditions`.
+ - Update `RangerOzoneAuthorizer` and request context handling to extract sensitive payload attributes and inject them into `RangerAccessRequest.getContext()`:
+ - `s3:object-lock-legal-hold`: Extracted from the request body (`ON` vs. `OFF`).
+ - `s3:object-lock-mode`: Extracted from the request body (`GOVERNANCE` vs. `COMPLIANCE`).
+ - `s3:x-amz-bypass-governance-retention`: Extracted from request headers (`true` vs. `false`).
+ - Implement runtime string matching within the evaluator to enforce value-level role segregation:
+ - Allow decoupling privileged operations (e.g., granting `PutObjectLegalHold` exclusively when `s3:object-lock-legal-hold == "ON"`, while restricting `OFF` to compliance officers).
+ - Prevent accidental or unauthorized lock-in by enforcing `s3:object-lock-mode == "GOVERNANCE"` for standard operators, denying `COMPLIANCE` configuration.
+
+3. **S3 Gateway & IAM Action Resolution**:
+ - Update `S3GAction` in `hadoop-ozone/s3gateway`: add audit actions for all lock configuration operations.
+ - Update `S3GActionIamMapper` in `hadoop-ozone/s3gateway`: map the new audit actions to standard IAM S3 actions (`GetBucketObjectLockConfiguration`, `PutBucketObjectLockConfiguration`, `GetObjectRetention`, `PutObjectRetention`, `GetObjectLegalHold`, `PutObjectLegalHold`, `BypassGovernanceRetention`).
+ - Update `IamSessionPolicyResolver.S3Action` in `hadoop-ozone/common`: register the actions with their resource scopes (Bucket vs. Object) and base permissions (`READ`, `WRITE`).
+ - Propagate `s3Action` via `S3Auth` through OM RPC to `RequestContext.s3Action` for evaluation by `RangerOzoneAuthorizer`.
Review Comment:
Following the discussion with @ChenSammi and @fmorg-git , OM does not set S3 actions itself to avoid duplicating the `S3GActionIamMapper` mapping logic.
More importantly, when requests come through the native OM Java API, we should assume the caller does not rely on S3-specific fine-grained action designs. Therefore, we use base access types (`READ_ACL` / `WRITE_ACL`) to cover these cases, or we can consider introducing dedicated access types if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4278985247
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
Thanks @fmorg-git, @ChenSammi, and @Gargi-jais11 for the great discussion!
I agree with @fmorg-git that setting S3 actions inside OM requests duplicates the mapping logic in `S3GActionIamMapper` and makes maintenance harder.
More importantly, from an architectural standpoint, OM should not be responsible for injecting S3 actions: **S3 actions are strictly an S3-protocol concept and should only be enforced when requests go through S3 Gateway.** For users calling the Ozone Native Java API or CLI directly, we should assume by default that they are operating under Ozone's native permission model rather than S3's fine-grained IAM action model.
Therefore, I propose we divide the authorization model into two clear tiers in the design doc:
1. **Strict S3-Compatible Fine-Grained Mode (S3G with Both Flags Enabled)**:
- We will explicitly document that fine-grained S3 action-level authorization (`PutObjectLegalHold`, `PutObjectRetention`, `BypassGovernanceRetention`, etc.) applies strictly to requests routed through **S3 Gateway** and requires **both** `ozone.s3g.sts.http.enabled=true` and `ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true`.
- In this mode, S3G extracts and passes the `s3Action` via `RequestContext` as it currently does, keeping the S3 action mapping cleanly isolated within S3G without extra overhead when the flag is off.
2. **Fallback / Native API Mode (Native Java API, CLI, Native ACL, or when either flag is disabled)**:
- Callers using the Ozone Native Java API/CLI (or when S3 action matching is not enabled) do not use S3 fine-grained actions; instead, authorization falls back to the base ACL permission table in OM.
- To prevent the privilege escalation issue @Gargi-jais11 pointed out (where a user with plain `WRITE` could remove a lock or retention), we should elevate the required base permissions for Object Lock management operations from plain `READ` / `WRITE` to **`READ_ACL` / `WRITE_ACL`** (e.g., querying lock/retention status requires `READ_ACL`, and configuring/clearing legal hold, retention, bucket lock config, or bypassing governance requires `WRITE_ACL`).
This way:
- Architectural boundaries remain clean: S3G handles S3 actions, and OM handles Ozone base permissions without hardcoding S3-specific concepts.
- Normal users with only `READ`/`WRITE`/`DELETE` data permissions cannot tamper with locks via Native Client or when STS is off.
- Clusters using Native ACLs (without Ranger) automatically get sensible admin-level protection for Object Lock out of the box.
- (We just need to make sure the base permission mappings for these new actions in Ranger / `IamSessionPolicyResolver` align with `READ_ACL` / `WRITE_ACL`).
How does this sound to everyone?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4321053613
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,589 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets, with authorization through Ozone native ACLs or Apache Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in Ozone is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design supports both Ozone native ACLs and Apache Ranger to manage Object Lock permissions on OBS buckets.
+Native ACLs are used by deployments without Ranger and allow the core logic to be implemented and tested independently of the Ranger integration.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, with permission and access control through native ACLs or Ranger.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Value-Level Ranger Authorization: a payload-aware Ranger condition evaluator (for example, allowing a legal hold to be set `ON` but not `OFF`) is deferred to a future phase.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Authorization: Ozone has no root account.
+ * With native ACLs, changing Object Lock settings requires `WRITE_ACL` and bypassing Governance retention is limited to Ozone administrators (see [Native ACL Mapping](#1-native-acl-mapping)).
+ * With Ranger, AWS-style action-level authorization for S3 requests requires both `ozone.s3g.sts.http.enabled` and `enableActionMatcherInPoliciesCondition` to be enabled; both are `false` by default (see [Prerequisite for S3 Fine-Grained Authorization](#3-prerequisite-for-s3-fine-grained-authorization)).
+ * Otherwise, and for native clients, Ranger falls back to `READ_ACL` / `WRITE_ACL`, and bypassing Governance retention requires `WRITE_ACL`, `WRITE` and `DELETE` (see [Fallback: Base Permission Mode](#4-fallback-base-permission-mode)).
+ * A Compliance retention cannot be removed through any API; it can only be changed offline with `ozone repair` (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until a principal allowed to change the legal hold (`WRITE_ACL` on the key, plus the `PutObjectLegalHold` action in Ranger Fine-Grained Mode) explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+* **Usage**:
+ * Used to preserve data for litigation, investigations or audits, when it is not known in advance how long the data must be kept.
+ * A legal hold has no mode and no expiration date. It is independent of retention: a key can have a legal hold, a retention period, or both,
+ and it stays locked until the legal hold is removed and the retention period has expired.
+ * A legal hold can be placed on a key in an Object Lock enabled bucket even if the key has no retention.
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * The bucket default can have both `rule` and `eventHold`. In that case a new key gets `retentionDate` = creation time + `rule` and an active event hold;
+ the key stays locked until the event hold is released and `retentionDate` has passed. With only `eventHold`, `retentionDate` is unset until the hold is released.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object / Head Object
+ * Get / Put Object Tagging
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the latest of the existing `retentionDate`, the requested `RetainUntilDate`, and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **authorization (native ACLs or Ranger)** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Base Permissions**: Object Lock operations are checked against the existing `READ_ACL` / `WRITE_ACL` types for both native ACLs and Ranger, so no new ACL type is needed (see below).
+* **Fine-Grained S3 Action Matching (Ranger)**: For requests through S3 Gateway, Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Native ACL Mapping
+
+Changing Object Lock state is treated as a permission change, so it requires `WRITE_ACL` rather than `WRITE`.
+A user who can only write or delete data cannot remove or shorten a lock.
+
+| Operation | Resource Level | Required Native ACL |
+|:---------------------------------------|:---------------|:-----------------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ_ACL` |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE_ACL` |
+| `GetObjectRetention` | Key | `READ_ACL` |
+| `PutObjectRetention` | Key | `WRITE_ACL` |
+| `GetObjectLegalHold` | Key | `READ_ACL` |
+| `PutObjectLegalHold` | Key | `WRITE_ACL` |
+| Bypass Governance retention | Key | Ozone administrator only |
+
+**Ozone administrator** means a user listed in `ozone.administrators` or a member of a group listed in `ozone.administrators.groups`
+(if neither is set, only the user that starts the OM is an administrator). This is the same check used for other admin-only OM operations.
+* The native authorizer already grants every ACL to Ozone administrators, so they also pass the `READ_ACL` / `WRITE_ACL` checks above.
+* Bypass is limited to administrators because the native ACL model has no permission dedicated to it: any ACL type would also be grantable by bucket or key owners,
+ who could then remove the Governance protection from their own data.
+* Read-only administrators (`ozone.readonly.administrators`) and S3 administrators (`ozone.s3.administrators`) cannot bypass Governance retention.
+* The administrator role is cluster wide, so it cannot be scoped to a single volume or bucket. Deployments that need per-bucket bypass entitlement should use Ranger.
+* The bypass does not apply to Compliance retention or a legal hold; a legal hold must be removed explicitly with `PutObjectLegalHold`, which an administrator can do because it only requires `WRITE_ACL`.
+
+---
+
+#### 2. Ranger Authorization Modes
+
+With Ranger, every Object Lock request is authorized in one of two modes.
+S3 actions are an S3 protocol concept, so only S3 Gateway sets them; OM does not add S3 actions to requests itself.
+In both modes OM always checks the base access type (`READ_ACL` / `WRITE_ACL`), and the OM WORM state validation is the same.
+
+| Request source | `ozone.s3g.sts.http.enabled` | `enableActionMatcherInPoliciesCondition` | Mode |
+|:--------------------------------|:-----------------------------|:-----------------------------------------|:------------------------------------------------------------------------|
+| S3 Gateway | `true` | `true` | [Fine-Grained Mode](#3-prerequisite-for-s3-fine-grained-authorization) |
+| S3 Gateway | `false` | `true` | [Base Permission Mode](#4-fallback-base-permission-mode); denied by policy items with an action condition (see below) |
+| S3 Gateway | `false` | `false` | [Base Permission Mode](#4-fallback-base-permission-mode) |
+| S3 Gateway | `true` | `false` | [Base Permission Mode](#4-fallback-base-permission-mode) (not recommended, see below) |
+| Native Java API / CLI | any | any | [Base Permission Mode](#4-fallback-base-permission-mode); with the Ranger flag enabled, denied by policy items with an action condition |
+
+* **Fine-Grained Mode**: Ranger checks the S3 action (for example `PutObjectLegalHold`) and its base access type in the [mapping below](#6-action-to-ranger-access-type-mapping).
+* **Base Permission Mode**: Ranger checks only the base access type.
+
+**Requests without an action when the Ranger flag is enabled**:
+
+When `enableActionMatcherInPoliciesCondition=true`, the result for a request without an S3 action (S3 Gateway with the STS flag disabled, or a native client) depends on the matching Ranger policy item:
+
+| Matching policy item | Request with an S3 action | Request without an S3 action |
+|:----------------------------------------------|:---------------------------------------------------|:-----------------------------------------------------|
+| Has an action condition (e.g. `PutObjectLegalHold`) | Granted only if the action is listed and the access type matches | **Denied**: the action condition cannot be satisfied |
+| Has no action condition | Granted if the access type matches | Granted if the access type matches (Base Permission Mode) |
+
+* If the principal is granted Object Lock access only through policy items with an action condition, every Object Lock request without an action is denied,
+ including the requests that the same principal could send through S3 Gateway with both flags enabled.
+* If the principal is granted the access type through a policy item without an action condition, the request falls back to Base Permission Mode.
+* When `enableActionMatcherInPoliciesCondition=false`, action conditions are not evaluated at all, so every request is in Base Permission Mode.
+
+---
+
+#### 3. Prerequisite for S3 Fine-Grained Authorization
+
+> _**Important for S3 users**_: AWS-style action-level authorization for Object Lock (separate permissions for `PutObjectRetention`,
+> `PutObjectLegalHold`, `PutBucketObjectLockConfiguration` and `BypassGovernanceRetention`) is only available when **both** flags below are enabled.
+> Both are `false` by default, so a cluster with default settings authorizes S3 Object Lock requests in [Base Permission Mode](#4-fallback-base-permission-mode).
+
+```properties
+# S3 Gateway and OM: extract the S3 action and forward it to the authorizer
+ozone.s3g.sts.http.enabled=true
+# Ranger Ozone service definition: evaluate the S3 action in policy conditions
+ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+```
+
+**Why both flags are required**:
+* `ozone.s3g.sts.http.enabled` decides whether the S3 action reaches Ranger at all. Today S3 Gateway sets `S3Authentication.s3Action` and OM copies it into the `RequestContext` only when this flag is `true`
+ (`EndpointBase.applyS3Action`, `OmMetadataReader.maybeAddToContextFromThreadLocal`). The action was originally only needed for STS, which is why it is tied to this flag.
+* `enableActionMatcherInPoliciesCondition` decides whether Ranger uses the action. When it is `false`, Ranger's `RangerActionMatcher` ignores the action and only checks the access type.
+* If only the Ranger flag is enabled, S3 Gateway sends no action. Requests are denied by policy items with an action condition, and fall back to Base Permission Mode only through policy items without one
+ (see [Requests without an action when the Ranger flag is enabled](#2-ranger-authorization-modes)). S3 users granted Object Lock permissions by action would therefore be denied, so this combination is not recommended either.
+* If only the STS flag is enabled, Ranger ignores the action, so requests also fall back to Base Permission Mode.
+ This combination is not recommended: STS session policies restrict temporary credentials by S3 action (see [AWS STS Design for Ozone S3](ozone-sts.md)),
+ so without the Ranger flag those restrictions would not be enforced. This is also why the STS flag is not enabled by default.
+
+**What S3 users get in Fine-Grained Mode**:
+* Object Lock permissions can be granted separately, as in AWS IAM. For example, a legal team can be granted `PutObjectLegalHold` without `PutObjectRetention`,
+ and `BypassGovernanceRetention` can be granted to a smaller group than the one that sets retention.
+* STS session policies (AssumeRole) can include or exclude Object Lock actions, and `IamSessionPolicyResolver` restricts the temporary credential accordingly.
+* Each action still requires its base access type, so a wrong action mapping in S3 Gateway cannot reduce the check below `READ_ACL` / `WRITE_ACL`.
+
+**Operator guidance**:
+* Clusters that expose Object Lock to S3 users with different roles (for example, data writers, compliance officers and administrators) should enable both flags before enabling Object Lock on any bucket.
+* The STS flag only affects S3 Gateway requests. Native clients never carry an action, so they are denied by policy items with an action condition and are granted only through policy items without one (see [below](#4-fallback-base-permission-mode)).
+* The user documentation of this feature will describe both flags and the difference between the two modes.
+
+---
+
+#### 4. Fallback: Base Permission Mode
+
+Base Permission Mode is used for every request without an S3 action:
+requests from the native Java API or CLI, and S3 Gateway requests when either flag in the [previous section](#3-prerequisite-for-s3-fine-grained-authorization) is disabled.
+OM does not set an S3 action for these requests, so that the S3 action mapping stays in `S3GActionIamMapper` only.
+When the Ranger flag is enabled, these requests are granted only through policy items without an action condition
+(see [Requests without an action when the Ranger flag is enabled](#2-ranger-authorization-modes)).
+
+**Checks**:
+
+| Operation | Resource Level | Required Ranger Access Type |
+|:---------------------------------------|:---------------|:----------------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ_ACL` |
+| `PutBucketObjectLockConfiguration` | Bucket | `WRITE_ACL` |
+| `GetObjectRetention` | Key | `READ_ACL` |
+| `PutObjectRetention` | Key | `WRITE_ACL` |
+| `GetObjectLegalHold` | Key | `READ_ACL` |
+| `PutObjectLegalHold` | Key | `WRITE_ACL` |
+| Bypass Governance retention | Key | `WRITE_ACL`, `WRITE` and `DELETE` |
+
+**Why `READ_ACL` / `WRITE_ACL` instead of `READ` / `WRITE`**:
+* Without an S3 action, a `WRITE` check could not tell `PutObject` from `PutObjectLegalHold`, so any user who can write a key could also remove its legal hold or shorten its retention.
+* Changing a lock is treated as a permission change, so it uses the same access type as changing an ACL. Users with only `READ`, `WRITE`, `CREATE` or `DELETE` on a key cannot read or change its Object Lock settings.
+* This matches the [Native ACL Mapping](#1-native-acl-mapping), so the same operation needs the same access type with native ACLs and with Ranger.
+
+**Limitations compared with Fine-Grained Mode**:
+* All Object Lock changes on a resource need the same `WRITE_ACL`, so they cannot be granted separately. A principal that can set a legal hold can also set retention on the same key, and can bypass Governance retention if it also has `WRITE` and `DELETE`.
+* Governance retention therefore only protects keys from principals without `WRITE_ACL`, `WRITE` and `DELETE`. Deployments that need a separate bypass entitlement should use Fine-Grained Mode.
+* `WRITE_ACL` is also the access type for changing ACLs, so granting Object Lock management also grants that.
+* Unlike native ACLs, where bypass is limited to Ozone administrators, Ranger has no Ozone administrator role, so bypass is granted through a policy with `WRITE_ACL`, `WRITE` and `DELETE`.
+* Compliance retention is not affected: it cannot be removed in either mode.
+
+**Native clients when both flags are enabled**:
+* Native clients carry no S3 action. A policy item with an action condition cannot be satisfied by a request without an action, so it does not grant the request.
+* A principal whose `WRITE_ACL` is limited to `PutObjectLegalHold` through S3 Gateway is therefore denied all Object Lock operations through a native client;
+ it cannot use a native client to get around the action-level restrictions.
+* To let a principal manage Object Lock through a native client, grant the access type through a policy item without an action condition.
+ That item applies to S3 Gateway requests as well, so the principal gets every Object Lock operation allowed by the access type, without action-level separation.
+
+---
+
+#### 5. Ranger & S3 Gateway Component Updates
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (`READ_ACL`, `WRITE_ACL`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings, mapped to the same `READ_ACL` / `WRITE_ACL` base access types.
+
+---
+
+#### 6. Action to Ranger Access Type Mapping
+
+In Fine-Grained Mode, following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ_ACL` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE_ACL` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ_ACL` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE_ACL` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ_ACL` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE_ACL` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE_ACL`, `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 7. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable through the Ozone API. **No role, administrator, native ACL or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires. The only way to change it early is the offline `ozone repair` command described below.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Bypass Entitlement**: With native ACLs, the principal must be an Ozone administrator (`ozone.administrators`).
+ With Ranger, the principal needs `WRITE_ACL`, `WRITE` and `DELETE` on the target key, and in Fine-Grained Mode also the **`BypassGovernanceRetention`** action.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted through the Ozone API by any admin, native ACL or Ranger policy until the retention period expires.
+> It can only be changed offline with `ozone repair` (see below).
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+* Bucket links: S3 buckets are often links (for example `/s3v/bucket1` → `/vol1/bucket1`). Unlike `OMBucketSetPropertyRequest`, which rejects property changes on links,
+ `PutBucketObjectLockConfiguration` follows the link and stores the configuration on the source bucket, and `GetBucketObjectLockConfiguration` reads it from there.
+ Authorization is checked on both the link and the source bucket, like other requests that resolve links. Key operations already resolve links, so WORM checks always use the source bucket's configuration.
+
+**Object-Level APIs:**
Review Comment:
Thanks for pointing that out. I have updated both tables to clarify this, including the dependent permissions for Ranger and the extra checks for native ACLs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266821265
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
Review Comment:
Let's change this to BucketRetentionConfig, to distinguish with KeyRetentionConfig.
Can this have both rule and eventHold, or they are one of relationship?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4269600238
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
Review Comment:
Is this Ranger flag enough on its own? In the current code, OM only passes s3Action to Ranger when `ozone.s3g.sts.http.enabled=true `, and that flag is false by default.
Supose a cluster has Ranger and the action-matcher flag on, but STS `off`. A user calls `PutObjectLegalHold` with `Status=OFF`. Ranger never sees the action, so it only checks plain WRITE on the key, and the hold is removed. Doesn't this mean that the STS flag is a prerequisite too?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
Review Comment:
I want some clarification on this: Will the lifecycle service retry the same locked keys on every run? or it skips locked keys when lifecycle picks its candidates.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
+one by one from the client, so it is blocked by Object Lock as well. Ozone has no account concept, so unlike AWS
+(where closing the account destroys all data, including Compliance-locked objects), there would be no way to decommission
+a tenant or volume until every retention period expires. An admin-only command is provided for this case,
+with the volume as the unit of deletion, as the closest analogue to deleting an AWS account:
+
+```text
+ozone admin om volume force-delete <volume>
+```
+
+* **Disabled by default**: guarded by a new OM config (for example `ozone.om.object.lock.volume.force.delete.enabled`, default `false`).
+ Deployments that must never allow removal of Compliance-locked data keep it disabled.
+* **Authorization**: OM administrators only (`ozone.administrators`). It is not exposed through S3 Gateway and is not covered by `BypassGovernanceRetention`.
+* **Execution**: OM marks the volume as deleting and rejects any new request on it. A background service then purges buckets and keys in batches
+ (including open keys, multipart uploads and snapshots), because a single Ratis transaction cannot delete a large volume. WORM checks are skipped only for this purge.
+ Deletion is irreversible.
+* **Audit**: the request and the purge result are written to the OM audit log.
+* **Trusted boundary**: this is the only way to remove Compliance-locked data through the Ozone API. The data is protected from everyone except a cluster administrator who explicitly enabled and ran this command.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+The additional metadata lookups are expected to have a small impact on throughput and latency; this will be verified with benchmarks and metrics during implementation.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions.
+
+By leveraging the **STS action-matching framework**, Ozone maps standard S3 API actions (e.g., `BypassGovernanceRetention`, `PutObjectLegalHold`) directly to standard Ranger primitive access types (e.g., `WRITE` and `READ`). This fine-grained, action-level authorization enforces the Principle of Least Privilege **without requiring custom dedicated access types**. It effectively prevents unauthorized tampering or removal of locks, ensuring robust compliance and data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+* **System Clock Tampering**: Artificially manipulating or rolling back the system clock (NTP/OS time) on OM or DataNode hosts to prematurely expire object retention periods or evade time-based WORM validation.
+* **Ranger Super Admin Privilege Abuse**: Modifying, disabling, or overriding Ozone authorization policies via Apache Ranger admin credentials (or direct access to Ranger’s backend database), allowing privileged operators to alter bucket WORM flags or grant unrestricted deletion permissions outside normal enforcement workflows.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+* **Table Schema Changes**: Since all newly introduced fields in the Bucket Table (`objectLockEnabled`, `defaultRetention`) and Key Table (`retention`, `legalHold`) are defined as `optional` in Protobuf, they are wire-compatible.
+ To avoid RocksDB serialization problems if the cluster is rolled back after new fields were written, a new OM layout feature gates Object Lock: until the upgrade is finalized, OM rejects every Object Lock request (bucket configuration, retention, legal hold, and `x-amz-object-lock-*` headers on Put / Multipart Upload), so no Object Lock metadata is persisted before finalization.
+* **OM Version Leveling**: OM version leveling is introduced primarily because this feature adds new OM APIs (such as `setRetentionConfig` and `setLegalHold`). The new OM version allows clients to determine whether the target OM cluster supports executing these Object Lock APIs during rolling upgrades.
+
+## Alternatives
+
+### 1. Why Not Use Native ACLs for Object Lock Management?
+
+* **Proposal**:
+ Support Native ACLs alongside or instead of Ranger to configure and manage Object Lock states (e.g., using Ozone native ACLs or POSIX-like ACLs at the Volume/Bucket/Key level).
+
+* **Why Rejected**:
+ 1. **Conflicting Security & Authorization Philosophies**:
+ Native ACLs in Ozone are primarily designed for granular, resource-centric permissions (often granting permissions directly to specific objects/buckets or inherited via path hierarchies). In contrast, Object Lock is an enterprise compliance feature that requires strict, role-based access governance (RBAC/ABAC). Compliance controls like WORM and Legal Holds are inherently tied to compliance officers, security auditors, or legal teams—not individual data owners.
+ 2. **Risk of Decentralized & Inconsistent Governance**:
+ Native ACLs allow object creators or bucket owners to adjust permissions. If lock controls were exposed through Native ACLs, standard bucket administrators or key owners could modify or override lock-related ACLs locally. Centralizing lock authorization exclusively in **Apache Ranger** ensures that compliance policies are managed from a unified pane of glass, preventing local policy drift or accidental misconfigurations.
+ 3. **Operational Complexity**:
+ Supporting dual authorization models (Ranger + Native ACLs) for critical compliance features introduces unnecessary complexity, potential security loopholes, and ambiguous resolution hierarchies during audits. Excluding Native ACLs ensures clean boundary enforcement.
+
+## Plan
+
+The implementation of S3 Object Lock in Apache Ozone is divided into six structured phases. This ensures incremental delivery, clean service boundaries, and testability at each stage.
+
+### Phase 1: Metadata Infrastructure & Schema Definitions
+
+*Goal: Establish protobuf schemas, in-memory domain models, and RocksDB table serialization codecs.*
+
+1. **Protobuf Definitions (`OmClientProtocol.proto`)**:
+ - Introduce retention domain messages:
+ - `RetentionMode` enum (`GOVERNANCE = 1`, `COMPLIANCE = 2`).
+ - `Rule` message (`retentionMode`, `timeUnit`, `duration`).
+ - `EventHold` message (`enabled`, `rule`).
+ - `RetentionConfig` message (`retentionMode`, `rule`, `eventHold`).
+ - `Retention` message (`retentionMode`, `retentionDate`, `eventHold`).
+ - Extend `BucketInfo` (OBS buckets only):
+ - `optional bool objectLockEnabled = 24 [default = false];`
+ - `optional RetentionConfig defaultRetention = 25;`
+ - Extend `KeyInfo` & `KeyInfoProtoLight`:
+ - `optional Retention retention = 23;`
+ - `optional bool legalHold = 24 [default = false];`
+
+2. **Domain Models & Helpers (`hadoop-ozone/common`)**:
+ - Update `OmBucketInfo` / `OmBucketInfo.Builder` with getters, setters, and protobuf translation.
+ - Update `OmKeyInfo` / `OmKeyInfo.Builder` with getters, setters, and protobuf translation.
+ - Introduce new class `Retention` to encapsulate bucket rules of retention:
+ - `fromProto` to convert from protobuf to java class.
+ - `toProto` to convert from java class to protobuf.
+ - Introduce `RetentionUtil` to provide utility methods for retention calculations:
+ - Check if `retention` (`retentionDate`, `eventHold`) is still locking: a key is locked if `legalHold` is true, `eventHold.enabled` is true, or `retentionDate` is later than the request time stamped by the leader in `preExecute` (not the local clock of each OM).
+ - Validate retention rules (e.g., duration > 0, valid time unit, etc.).
+
+---
+
+### Phase 2: Ranger Integration & Action Mapping
+
+*Goal: Enable fine-grained access control separating standard data mutating permissions from compliance actions.*
+
+1. **Ranger Action Mapping & UI Validation (`ozone.json`)**:
+ - Do **NOT** create new underlying Ranger access types. Retain primitive access types (`READ`, `WRITE`, `DELETE`) and leverage the STS action-matching framework (`action-matches`).
+ - Register the 7 S3 Object Lock actions in `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json`:
+ - `GetBucketObjectLockConfiguration` (Bucket level: requires `READ`)
+ - `PutBucketObjectLockConfiguration` (Bucket level: requires `WRITE`)
+ - `GetObjectRetention` (Key level: requires `READ`)
+ - `PutObjectRetention` (Key level: requires `WRITE`)
+ - `GetObjectLegalHold` (Key level: requires `READ`)
+ - `PutObjectLegalHold` (Key level: requires `WRITE`)
+ - `BypassGovernanceRetention` (Key level: requires `WRITE` / `DELETE`)
+ - Enable design-time validation in Ranger React UI to ensure administrators cannot grant compliance actions without selecting their required primitive permissions.
+
+2. **Payload-Aware String Matcher Condition Evaluator**:
+ - Register a dedicated custom condition evaluator (e.g., `RangerOzoneValueMatcher`, extending `RangerAbstractConditionEvaluator`) in `ranger-servicedef-ozone.json` under `policyConditions`.
+ - Update `RangerOzoneAuthorizer` and request context handling to extract sensitive payload attributes and inject them into `RangerAccessRequest.getContext()`:
+ - `s3:object-lock-legal-hold`: Extracted from the request body (`ON` vs. `OFF`).
+ - `s3:object-lock-mode`: Extracted from the request body (`GOVERNANCE` vs. `COMPLIANCE`).
+ - `s3:x-amz-bypass-governance-retention`: Extracted from request headers (`true` vs. `false`).
+ - Implement runtime string matching within the evaluator to enforce value-level role segregation:
+ - Allow decoupling privileged operations (e.g., granting `PutObjectLegalHold` exclusively when `s3:object-lock-legal-hold == "ON"`, while restricting `OFF` to compliance officers).
+ - Prevent accidental or unauthorized lock-in by enforcing `s3:object-lock-mode == "GOVERNANCE"` for standard operators, denying `COMPLIANCE` configuration.
+
+3. **S3 Gateway & IAM Action Resolution**:
+ - Update `S3GAction` in `hadoop-ozone/s3gateway`: add audit actions for all lock configuration operations.
+ - Update `S3GActionIamMapper` in `hadoop-ozone/s3gateway`: map the new audit actions to standard IAM S3 actions (`GetBucketObjectLockConfiguration`, `PutBucketObjectLockConfiguration`, `GetObjectRetention`, `PutObjectRetention`, `GetObjectLegalHold`, `PutObjectLegalHold`, `BypassGovernanceRetention`).
+ - Update `IamSessionPolicyResolver.S3Action` in `hadoop-ozone/common`: register the actions with their resource scopes (Bucket vs. Object) and base permissions (`READ`, `WRITE`).
+ - Propagate `s3Action` via `S3Auth` through OM RPC to `RequestContext.s3Action` for evaluation by `RangerOzoneAuthorizer`.
Review Comment:
should OM decide the S3 action for the new Object Lock requests itself, instead of using the s3Action string from S3 Gateway?
Today OM passes `S3Authentication.s3Action` to Ranger as it is, and never checks it against the request type. This leaves three gaps:
1. If S3 Gateway labels a legal-hold call as PutObject by mistake (for example, a missed mapping in S3GActionIamMapper), Ranger checks PutObject while OM removes the legal hold.
2. Native clients (bucket.setLegalHold(key, false)) send no label, so Ranger only checks plain WRITE.
3. wiith ozone.s3g.sts.http.enabled=false (the default), the label is dropped, so again only plain WRITE is checked.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
+one by one from the client, so it is blocked by Object Lock as well. Ozone has no account concept, so unlike AWS
+(where closing the account destroys all data, including Compliance-locked objects), there would be no way to decommission
+a tenant or volume until every retention period expires. An admin-only command is provided for this case,
+with the volume as the unit of deletion, as the closest analogue to deleting an AWS account:
+
+```text
+ozone admin om volume force-delete <volume>
+```
+
+* **Disabled by default**: guarded by a new OM config (for example `ozone.om.object.lock.volume.force.delete.enabled`, default `false`).
+ Deployments that must never allow removal of Compliance-locked data keep it disabled.
+* **Authorization**: OM administrators only (`ozone.administrators`). It is not exposed through S3 Gateway and is not covered by `BypassGovernanceRetention`.
+* **Execution**: OM marks the volume as deleting and rejects any new request on it. A background service then purges buckets and keys in batches
+ (including open keys, multipart uploads and snapshots), because a single Ratis transaction cannot delete a large volume. WORM checks are skipped only for this purge.
+ Deletion is irreversible.
+* **Audit**: the request and the purge result are written to the OM audit log.
+* **Trusted boundary**: this is the only way to remove Compliance-locked data through the Ozone API. The data is protected from everyone except a cluster administrator who explicitly enabled and ran this command.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+The additional metadata lookups are expected to have a small impact on throughput and latency; this will be verified with benchmarks and metrics during implementation.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions.
+
+By leveraging the **STS action-matching framework**, Ozone maps standard S3 API actions (e.g., `BypassGovernanceRetention`, `PutObjectLegalHold`) directly to standard Ranger primitive access types (e.g., `WRITE` and `READ`). This fine-grained, action-level authorization enforces the Principle of Least Privilege **without requiring custom dedicated access types**. It effectively prevents unauthorized tampering or removal of locks, ensuring robust compliance and data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+* **System Clock Tampering**: Artificially manipulating or rolling back the system clock (NTP/OS time) on OM or DataNode hosts to prematurely expire object retention periods or evade time-based WORM validation.
+* **Ranger Super Admin Privilege Abuse**: Modifying, disabling, or overriding Ozone authorization policies via Apache Ranger admin credentials (or direct access to Ranger’s backend database), allowing privileged operators to alter bucket WORM flags or grant unrestricted deletion permissions outside normal enforcement workflows.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+* **Table Schema Changes**: Since all newly introduced fields in the Bucket Table (`objectLockEnabled`, `defaultRetention`) and Key Table (`retention`, `legalHold`) are defined as `optional` in Protobuf, they are wire-compatible.
+ To avoid RocksDB serialization problems if the cluster is rolled back after new fields were written, a new OM layout feature gates Object Lock: until the upgrade is finalized, OM rejects every Object Lock request (bucket configuration, retention, legal hold, and `x-amz-object-lock-*` headers on Put / Multipart Upload), so no Object Lock metadata is persisted before finalization.
+* **OM Version Leveling**: OM version leveling is introduced primarily because this feature adds new OM APIs (such as `setRetentionConfig` and `setLegalHold`). The new OM version allows clients to determine whether the target OM cluster supports executing these Object Lock APIs during rolling upgrades.
+
+## Alternatives
+
+### 1. Why Not Use Native ACLs for Object Lock Management?
+
+* **Proposal**:
+ Support Native ACLs alongside or instead of Ranger to configure and manage Object Lock states (e.g., using Ozone native ACLs or POSIX-like ACLs at the Volume/Bucket/Key level).
+
+* **Why Rejected**:
+ 1. **Conflicting Security & Authorization Philosophies**:
+ Native ACLs in Ozone are primarily designed for granular, resource-centric permissions (often granting permissions directly to specific objects/buckets or inherited via path hierarchies). In contrast, Object Lock is an enterprise compliance feature that requires strict, role-based access governance (RBAC/ABAC). Compliance controls like WORM and Legal Holds are inherently tied to compliance officers, security auditors, or legal teams—not individual data owners.
+ 2. **Risk of Decentralized & Inconsistent Governance**:
+ Native ACLs allow object creators or bucket owners to adjust permissions. If lock controls were exposed through Native ACLs, standard bucket administrators or key owners could modify or override lock-related ACLs locally. Centralizing lock authorization exclusively in **Apache Ranger** ensures that compliance policies are managed from a unified pane of glass, preventing local policy drift or accidental misconfigurations.
+ 3. **Operational Complexity**:
+ Supporting dual authorization models (Ranger + Native ACLs) for critical compliance features introduces unnecessary complexity, potential security loopholes, and ambiguous resolution hierarchies during audits. Excluding Native ACLs ensures clean boundary enforcement.
+
+## Plan
+
+The implementation of S3 Object Lock in Apache Ozone is divided into six structured phases. This ensures incremental delivery, clean service boundaries, and testability at each stage.
+
+### Phase 1: Metadata Infrastructure & Schema Definitions
+
+*Goal: Establish protobuf schemas, in-memory domain models, and RocksDB table serialization codecs.*
+
+1. **Protobuf Definitions (`OmClientProtocol.proto`)**:
+ - Introduce retention domain messages:
+ - `RetentionMode` enum (`GOVERNANCE = 1`, `COMPLIANCE = 2`).
+ - `Rule` message (`retentionMode`, `timeUnit`, `duration`).
+ - `EventHold` message (`enabled`, `rule`).
+ - `RetentionConfig` message (`retentionMode`, `rule`, `eventHold`).
+ - `Retention` message (`retentionMode`, `retentionDate`, `eventHold`).
+ - Extend `BucketInfo` (OBS buckets only):
+ - `optional bool objectLockEnabled = 24 [default = false];`
+ - `optional RetentionConfig defaultRetention = 25;`
+ - Extend `KeyInfo` & `KeyInfoProtoLight`:
+ - `optional Retention retention = 23;`
+ - `optional bool legalHold = 24 [default = false];`
+
+2. **Domain Models & Helpers (`hadoop-ozone/common`)**:
+ - Update `OmBucketInfo` / `OmBucketInfo.Builder` with getters, setters, and protobuf translation.
+ - Update `OmKeyInfo` / `OmKeyInfo.Builder` with getters, setters, and protobuf translation.
+ - Introduce new class `Retention` to encapsulate bucket rules of retention:
+ - `fromProto` to convert from protobuf to java class.
+ - `toProto` to convert from java class to protobuf.
+ - Introduce `RetentionUtil` to provide utility methods for retention calculations:
+ - Check if `retention` (`retentionDate`, `eventHold`) is still locking: a key is locked if `legalHold` is true, `eventHold.enabled` is true, or `retentionDate` is later than the request time stamped by the leader in `preExecute` (not the local clock of each OM).
+ - Validate retention rules (e.g., duration > 0, valid time unit, etc.).
+
+---
+
+### Phase 2: Ranger Integration & Action Mapping
+
+*Goal: Enable fine-grained access control separating standard data mutating permissions from compliance actions.*
+
+1. **Ranger Action Mapping & UI Validation (`ozone.json`)**:
+ - Do **NOT** create new underlying Ranger access types. Retain primitive access types (`READ`, `WRITE`, `DELETE`) and leverage the STS action-matching framework (`action-matches`).
+ - Register the 7 S3 Object Lock actions in `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json`:
+ - `GetBucketObjectLockConfiguration` (Bucket level: requires `READ`)
+ - `PutBucketObjectLockConfiguration` (Bucket level: requires `WRITE`)
+ - `GetObjectRetention` (Key level: requires `READ`)
+ - `PutObjectRetention` (Key level: requires `WRITE`)
+ - `GetObjectLegalHold` (Key level: requires `READ`)
+ - `PutObjectLegalHold` (Key level: requires `WRITE`)
+ - `BypassGovernanceRetention` (Key level: requires `WRITE` / `DELETE`)
+ - Enable design-time validation in Ranger React UI to ensure administrators cannot grant compliance actions without selecting their required primitive permissions.
+
+2. **Payload-Aware String Matcher Condition Evaluator**:
+ - Register a dedicated custom condition evaluator (e.g., `RangerOzoneValueMatcher`, extending `RangerAbstractConditionEvaluator`) in `ranger-servicedef-ozone.json` under `policyConditions`.
+ - Update `RangerOzoneAuthorizer` and request context handling to extract sensitive payload attributes and inject them into `RangerAccessRequest.getContext()`:
+ - `s3:object-lock-legal-hold`: Extracted from the request body (`ON` vs. `OFF`).
+ - `s3:object-lock-mode`: Extracted from the request body (`GOVERNANCE` vs. `COMPLIANCE`).
+ - `s3:x-amz-bypass-governance-retention`: Extracted from request headers (`true` vs. `false`).
+ - Implement runtime string matching within the evaluator to enforce value-level role segregation:
+ - Allow decoupling privileged operations (e.g., granting `PutObjectLegalHold` exclusively when `s3:object-lock-legal-hold == "ON"`, while restricting `OFF` to compliance officers).
+ - Prevent accidental or unauthorized lock-in by enforcing `s3:object-lock-mode == "GOVERNANCE"` for standard operators, denying `COMPLIANCE` configuration.
+
+3. **S3 Gateway & IAM Action Resolution**:
+ - Update `S3GAction` in `hadoop-ozone/s3gateway`: add audit actions for all lock configuration operations.
+ - Update `S3GActionIamMapper` in `hadoop-ozone/s3gateway`: map the new audit actions to standard IAM S3 actions (`GetBucketObjectLockConfiguration`, `PutBucketObjectLockConfiguration`, `GetObjectRetention`, `PutObjectRetention`, `GetObjectLegalHold`, `PutObjectLegalHold`, `BypassGovernanceRetention`).
+ - Update `IamSessionPolicyResolver.S3Action` in `hadoop-ozone/common`: register the actions with their resource scopes (Bucket vs. Object) and base permissions (`READ`, `WRITE`).
+ - Propagate `s3Action` via `S3Auth` through OM RPC to `RequestContext.s3Action` for evaluation by `RangerOzoneAuthorizer`.
+
+---
+
+### Phase 3: Object Lock Management APIs
+
+*Goal: Implement S3 REST endpoints, OM client RPC protocols, and OM HA consensus handlers for configuring and querying locks.*
+
+#### 1. Bucket Object Lock Configuration
+- **`PutBucketObjectLockConfiguration`**:
+ - *S3G*: Route `PUT /{bucket}?object-lock`. Parse `ObjectLockConfiguration` XML (`ObjectLockEnabled`, optional `DefaultRetention`).
+ - *RPC*: Add `SetBucketObjectLockConfigRequest` / `Response` in `OmClientProtocol.proto`.
+ - *OM Handler (`OMBucketSetObjectLockConfigRequest`)*:
+ - `preExecute`: Validate Ranger action `PutBucketObjectLockConfiguration`. Fail-fast on invalid parameters.
+ - `validateAndUpdateCache`: Under bucket write lock, check bucket exists and layout is OBS. Ensure immutability: if `objectLockEnabled` is already `true`, it cannot be toggled to `false`. Set `objectLockEnabled` and update `defaultRetention`. Commit to `BucketTable`.
Review Comment:
Could u please help me out how does this work with bucket links? S3 buckets are often links (for example /s3v/bucket1 → /vol1/bucket1), and OMBucketSetPropertyRequest already rejects property changes on links.
Do we follow the link and store the config on vol1/bucket1, or reject the request?
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
Review Comment:
What about rename? OBS buckets support RenameKey and RenameKeys. S3 rename already returns NotImplemented, but the native path is open through `ozone sh key rename`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4356657966
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
Review Comment:
Can we rename this to EventHoldDuration?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4357230734
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,598 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
Review Comment:
There is one limitation about bucket locking default configuration,
`If you specify both a default retention period (Days or Years) and a DefaultEventHold, the event hold duration can't exceed the default retention period.
`
https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-properties-s3-bucket-defaultretention.html
@chungen0126 , I would suggest except the general Terminology section, we have a new section for bucket, and object, about which locking types are allowed for each other, for example, bucket doesn't have legal hold, and have retention duration instead of RetainUntilDate, while object has legal hold, RetainUntilDate and retention duration in EventHold.
And if both bucket, and object locking configuration are there, what final behavior will be. For example, for an object, if legal hold, RetainUntilDate and EventHold are all specified, or different combination are specified, what will be the final behavior?
Also except the statement description, if we can have the formula like following, it will help a lot for the understanding.
`Final RetainUntilDate = max(Existing RetainUntilDate, Release Timestamp + EventHoldDuration)`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4349812257
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
You're right, OM's administrator check does not depend on the authorizer, so "Ranger has no administrator role" was not the right reason.However, I'd still keep Compliance under Ranger policies instead of requiring an Ozone administrator:
1. A Compliance retention set by mistake is not stuck forever: `ozone repair om object-lock` can remove or downgrade it offline.
2. `ozone.administrators` is cluster wide, so anyone who manages Compliance for a single bucket would have to be an administrator of the whole cluster.
3. It would remove Ranger's fine-grained delegation, for example granting `PutObjectRetention` with `action-matches` to a compliance team on specific buckets only, which is the main reason to use Ranger for this feature.
For the risk of action matcher being off by default, we've highlighted it in the doc and will cover it in the user guide. The recommended mitigation remains enabling action-matches.
I've updated the doc with these reasons instead. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3987511891
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
Review Comment:
Thanks for pointing that out.
- Enabling at Creation: We can definitely support enabling Object Lock directly at bucket creation time (e.g., via the S3 CreateBucket header/parameters) to eliminate any compliance window. I updated the doc.
- One-way Flag: Exactly, objectLockEnabled is designed as a one-way flag—once enabled on a bucket, it cannot be cleared or disabled.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3986358284
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
Review Comment:
Object locking can be configured and managed via both the Ozone API and the S3 API.
- Ozone API: It uses OzoneBucket#setRetentionConfig. If no default retention is set, the rule field within retention will simply be null.
- S3 API: It is managed through PutBucketObjectLockConfiguration, which basically follows the standard S3 specifications. You can refer to the official [AWS S3 documentation](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObjectLockConfiguration.html) for more details.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
Review Comment:
Thanks for pointing this out!
You're right that it relies on the system clock. However, altering the system time requires OS-level root privileges. I've added a Trusted Boundary section to clarify that any operations requiring root access bypass the application-level API and are outside the trusted boundary.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
+ * **Enforcement Flow**: While regular users are strictly blocked from mutating locked objects in Governance Mode, any request attempting to overwrite or delete such objects requires explicit `BYPASS_GOVERNANCE` entitlement granted in the Ranger policy for that Volume. Without this explicit permission, even administrators using standard client tools cannot modify or remove the locked object.
+### New Ozone APIs
+
+**ObjectStore**
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(@Nullable RetentionArgs retentionArgs);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support the following standard AWS S3 Object Lock APIs:
+
+**Bucket-Level APIs:**
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Maps to the new Ozone API to enable Object Lock for an existing bucket. Configuring default retention rules (DefaultRetention) is optional.
+
+**Object-Level APIs:**
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the `RetainUntilDate`.
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
Review Comment:
> In a distributed development environment, it is possible for your application to initiate several updates on the same object at the same time. Your application might initiate several multipart uploads using the same object key.
AWS S3 explicitly allows multiple concurrent multipart uploads for the same key.
If we only reject requests at CreateMultipartUpload, an MPU initiated before the lock is applied could still be completed after the lock becomes active.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+#### Key Table
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+- Legal Hold: Introduces three new Access Types—GET_LEGAL_HOLD, PUT_LEGAL_HOLD, and CLEAR_LEGAL_HOLD—and adds them to accessTypeRestrictions for Keys.
+- Retention: Introduces the BYPASS_GOVERNANCE Access Type and adds it to accessTypeRestrictions for Volumes.
+
+### New Ozone APIs
+#### ObjectStore
+
+```java
+ public void addRetentionConfig(OzoneObj obj, RetentionArgs retentionArgs);
+ public void addLegalHold(OzoneObj obj, bool hold);
+```
+
+#### OzoneBucket
+
+```java
+public void enableObjectLock(boolean enableObjectLock);
+public void setRetentionConfig(RetentionArgs retentionArgs);
+```
+
+> _**Note**:
+> AWS S3 supports enabling Object Lock and configuring RetentionConfig directly during bucket creation.
+> In Apache Ozone's current design, these settings must be configured via dedicated API calls after the bucket has been created.
+
+
+### Impact on Write and Delete Flows
+#### Impact on Put Object Flow
+
+Put Flow:
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+#### Impact on Multipart Upload Flow
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. During Initiate Multipart Upload and Put Part phases, WORM checks are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. During the Complete Multipart Upload phase, final validation occurs in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+#### Impact on Delete Object Flow
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
Review Comment:
Bucket & Volume Removal:
1. In Ozone, a bucket cannot be deleted if it is non-empty (still contains keys/directories), and similarly, a volume cannot be deleted if it contains buckets.
2. As noted in Non-Goals, FSO buckets are excluded from this design:
- Target use case: S3 Object Lock is an S3-centric compliance feature targeted at OBS buckets.
- Directory operations: FSO deletes directory trees by moving them to the deletedDirectoryTable in $O(1)$ time without validating deep children. Checking WORM locks recursively would break this $O(1)$ guarantee and require expensive tree traversals.
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
Review Comment:
Updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3986400221
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ - Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial/Complete
+ - Set Retention Period
+- Allowed Operations:
+ - Get Object
+
+> _**Note**:
+> - Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> - Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+## Design
+
+### Table Changes
+
+#### Bucket Table
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
Review Comment:
Thanks for pointing this out!
You're right that it relies on the system clock. However, altering the system time requires OS-level root privileges. I've added a Trusted Boundary section to clarify that any operations requiring root access bypass the application-level API and are outside the trusted boundary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3987478893
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,198 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Background
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+## Goal & Non-Goal
+### Goal
+
+- Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+- Support single-version objects only.
+
+### Non-Goal
+
+- Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+- FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+- Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+### Terminology
+#### Legal Hold
+
+- Definition: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+- Restricted Operations:
+ - Put Object
+ - Delete Object
+ - Multipart Initial / Complete
+- Allowed Operations:
+ - Get Object
+ - Get Legal Hold
+ - Put Legal Hold (Depends on permission)
+
+#### Retention
+
+- Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+- Retention Modes:
+ - Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
Review Comment:
Thanks for pointing this out!
I've added a Trusted Boundary section to clarify that any operations requiring root access bypass the application-level API and are outside the trusted boundary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3955544108
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
+
+#### 2. Retention Access Control and Governance Mode Bypass
+
+For object retention, the interaction with Ranger policies depends on the configured Retention Mode:
+
+* **Compliance Mode**: Serves as the strictest compliance tier. Until the retention period expires, **no role or Ranger permission can bypass or overwrite the lock**, including cluster administrators.
+* **Governance Mode and `BYPASS_GOVERNANCE`**:
+ * **Mechanism**: Governance Mode allows authorized users to overwrite, delete, or alter the retention duration of a locked object before its expiration date.
+ * **Authorization Binding**: Introduces the **`BYPASS_GOVERNANCE`** Access Type, configured under `accessTypeRestrictions` at the **Volume level**.
Review Comment:
Why BYPASS_GOVERNANCE is configured at volume level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3954617490
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
Review Comment:
Thanks @chungen0126 for working on this design.
It looks like bucket level retention configuration is different than object level. It support days and years, not the retainUntilDate.
https://docs.aws.amazon.com/cli/latest/reference/s3api/put-object-lock-configuration.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3955707684
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
Review Comment:
@fapifta , would you mind sharing some knowledge with @chungen0126 about how to expose new permissions for Ranger, and what are the major tasks needed for Ranger integration, including UI changes?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3963561653
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
Review Comment:
I added comments
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r3964242643
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,291 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: accepted
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock)
+and legacy FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* FSO Legacy Buckets: Object Lock support for legacy FSO buckets is excluded.
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## 5. Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a fixed retention duration (specified in days or years) for an object and applies a specific retention mode.
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with bypassgovernance permissions can bypass restrictions to perform modifications or deletions.
+* Restricted Operations:
+ * Put Object
+ * Delete Object
+ * Multipart Initial/Complete
+ * Set Retention Period
+* Allowed Operations:
+ * Get Object
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ required bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ optional RetentionMode retentionMode = 23;
+ optional uint64 retainUntilDate = 24;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retentionConfig & legalHold.
+
+```protobuf
+ message KeyInfo {
+ // ... existing fields
+ optional RetentionConfig retentionConfig = 23;
+ optional bool legalHold = 24 [default = false];
+ }
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate mechanism combining **Ranger authorization checks** and **underlying WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a user is granted Ranger `WRITE` or `DELETE` permissions, the operation is immediately denied with a `403 Access Denied` (`WORMProtectionException`) if the target object is actively locked by an unexpired retention period or an active Legal Hold.
+* **Lock Management and Bypass Authorization**: To adhere to the Principle of The Least Privilege, dedicated Ranger Access Types are introduced to govern Legal Hold state transitions and Governance Mode bypass privileges:
+
+#### 1. Legal Hold Access Control
+
+Three dedicated Access Types are added to `accessTypeRestrictions` at the Key level to govern Legal Hold states:
+
+* **`GET_LEGAL_HOLD`**: Allows querying the current Legal Hold status of an object (maps to the `GetObjectLegalHold` API).
+* **`PUT_LEGAL_HOLD`**: Grants permission to apply a Legal Hold to an object (maps to `PutObjectLegalHold` with status ON). Once applied, the object is locked indefinitely, blocking all overwrite and delete operations.
+* **`CLEAR_LEGAL_HOLD`**: Grants permission to remove a Legal Hold from an object (maps to `PutObjectLegalHold` with status OFF). Because this clears immutability protection, it is treated as a high-privilege action restricted only to authorized users.
Review Comment:
Also, to use the actions in Ranger, ensure to set the Ranger feature flag to true: https://ozone.apache.org/docs/next/administrator-guide/operations/s3/sts#ranger-feature-flag-action-matches-policy-condition
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4397433671
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
Changed direction after the comment at https://github.com/apache/ozone/pull/11053#discussion_r4397427029 : the recovery path is now an administrator-only PutObjectRetention instead of the offline repair. See "Releasing a Compliance Retention".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4397433671
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
Changed direction after the comment at https://github.com/apache/ozone/pull/11053#discussion_r4397427029 : the recovery path is now an administrator-only PutObjectRetention instead of the offline repair. See "Relaxing a Compliance Retention".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4290230022
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
@chungen0126 , since we don't have these fine grained permissions in native ACL than Ranger, so instead of using READ_ACL or WRITE_ACL, I would like to propose,
for object and bucket
- READ permission for lock information receive
- owner for lock information update(except the compliance update action)
for compliance mode update
- Only ozone admin
And fallback is not supported, if Ranger is configured, when always use Ranger for the authorization; If Native ACL is configured, when always use Native ACL.
@symious @xichen01 @ivandika3 , will this meet your requirements if in future you guys want to use this feature?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4266852685
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
Review Comment:
I would suggest we defer and move this support to next phase. It's good to have. But from my understanding, FSO support is more important for this feature in next.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "fmorg-git (via GitHub)" <gi...@apache.org>.
fmorg-git commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4278186434
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
It was a good observation by @Gargi-jais11 that actions are only sent in S3Gateway and Ranger if STS is enabled (they were only needed for STS at that point). The reason STS flag was not defaulted to true is because the Ranger team defaulted their flag to false and if the STS flag is true and the Ranger flag is false, there would be security issues. However, I'm concerned that the approach outlined in this design would have actions being set in multiple places like S3gActionIamMapper and in these new requests like OMKeySetLegalHoldRequest so the maintenance becomes more difficult. Would it be feasible to separate the actions being sent in S3Gateway and RequestContext from the STS flag? @ChenSammi thoughts?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "ChenSammi (via GitHub)" <gi...@apache.org>.
ChenSammi commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4278869462
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
@fmorg-git , how about we always carry the S3 action in RequestContext during Ranger authentication? Whether these action will be used in Ranger will depends on ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition is true or not. The cost of this solution is we need to extract and set the S3 action in S3g, and pass to OM for each request, and if it's an expensive operation, then it may have performance impact.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4279683007
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,612 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design document aims to plan and implement the Object Lock mechanism for OBS buckets integrated with Ranger.
+The primary objective is to provide data immutability and tamper-proof protection through the object locking feature.
+
+## Problem statement
+
+With growing demands for data security and compliance, ensuring that critical data stored in OBS (Object Storage) is
+protected from accidental or malicious deletion and overwriting has become an essential system protection requirement.
+To establish a more rigorous data protection mechanism, we plan to introduce the Object Lock feature.
+
+Considering the current system architecture and access control strategies,
+this design integrates with existing Apache Ranger to manage Object Lock permissions on OBS buckets.
+Meanwhile, to accelerate core feature delivery, we have decided to exclude complex multi-version locking (Versioning Lock),
+Legacy buckets, and FSO buckets from this initial release. In addition, support for Native ACLs is excluded; Native ACLs typically grant permissions
+at the granular bucket or object level, whereas Object Lock permission management favors broad, role-based authorization,
+creating a conflict in design philosophies. Narrowing the scope allows us to focus on the core functionality and ensure a rapid,
+stable rollout of baseline tamper-proof protection.
+
+**Goal:**
+
+* Implement the Object Lock feature on standard OBS buckets, fully integrated with Ranger for permission and access control.
+* Support single-version objects only.
+
+## Non-Goal
+
+* Versioning Lock: Support for locking across multiple object versions is deferred (multi-version core features are currently under development).
+* Legacy and FSO Buckets: Object Lock support for legacy and FSO buckets is excluded. The following considerations are deferred to a future phase:
+ * Lock Granularity: FSO separates files and directories across FileTable and DirectoryTable, requiring explicit policy decisions on whether lock semantics apply at the file level, directory level, or both.
+ * Rename & Move Semantics: Renaming or moving paths alters namespace hierarchy without mutating object content; rules must be defined to either block operations on locked paths or preserve lock metadata across the same objectID
+* Native ACL Support: Native ACLs will not be used for access control or advanced configuration such as Retention Mode (Governance); access management is centralized exclusively via Ranger.
+
+## Differences from AWS
+
+* Single-Version Model: Locks apply to the current key rather than individual object versions.
+* Direct Overwrite Blocking: Unlike AWS S3 (which writes a new version), Ozone strictly rejects Put, Copy, and MPU requests on locked keys with 403 Access Denied (WORMProtectionException).
+* Ranger-Dependent Auth: Lock governance relies exclusively on Apache Ranger and STS action matching (enableActionMatcherInPoliciesCondition). Object Lock is not supported on clusters without Ranger: OM rejects all Object Lock configuration requests unless the Ranger authorizer is in use.
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* **Definition**: Applies an indefinite lock status to an object. The object remains protected until an administrator explicitly removes the lock (Remove Legal Hold).
+* **Restricted Operations**:
+ * Put Object
+ * Delete Object
+ * Multipart Initial / Complete
+* **Allowed Operations**:
+ * Get Object
+ * Get Legal Hold
+ * Put Legal Hold (Depends on permission)
+
+**Retention**
+
+* Definition: Configures a retention policy for an object to prevent deletion or modification. Retention can be duration-based (configured in days or years, establishing a fixed `RetainUntilDate`) or event-driven (**Event Hold / Event-based Retention**, where an object remains protected indefinitely until an external business or legal event triggers the final retention countdown).
+* Retention Modes:
+ * Compliance Mode: The strictest protection tier. Once applied, no user (including root/admin) can remove the lock, shorten the duration, or overwrite the object before the retention period expires.
+ * Governance Mode: A flexible protection tier. Standard users are restricted by locking rules, but users with `BypassGovernanceRetention` permissions can bypass restrictions to modify or delete the object.
+* Event Hold / Event-based Retention:
+ * Used for records management where the retention lifecycle depends on external events (e.g., contract termination, employee departure, loan closure).
+ * Keeps the object immutable while awaiting event notification; once the event occurs, the definitive expiration date (`RetainUntilDate`) is calculated and applied.
+ * While the event has not occurred, `retentionDate` is unset and `eventHold.enabled = true` keeps the key locked indefinitely. `eventHold.rule` is copied from the bucket default when the key is created, so later changes to the bucket default do not affect existing keys.
+ * While an event hold is active, a `GOVERNANCE` key can be deleted or overwritten with `bypassGovernanceRetention`; a `COMPLIANCE` key cannot.
+* Restricted Operations:
+ * Put Object / Copy Object (Overwrites)
+ * Delete Object
+ * Multipart Upload (Initial / Complete)
+ * Shorten Retention Period (in Compliance Mode)
+* Allowed Operations:
+ * Get Object
+ * Extend Retention Period
+
+#### Event Hold Trigger Workflow
+For enterprise governance workflows, the retention lifecycle often depends on specific business triggers rather than a static configuration at creation. The Event Hold mechanism handles this dynamic lifecycle through the following state transitions:
+
+```text
+object created
+ |
+event hold applied (indefinite WORM protection)
+ |
+business event occurs
+ |
+hold released
+ |
+retention timer starts (RetainUntilDate is calculated)
+ |
+WORM until timer expires
+```
+
+The "business event occurs" step is triggered with `PutObjectRetention`, so no new API or Ranger action is needed
+and S3 clients can release the hold through S3 Gateway (native clients use `OzoneBucket#setRetention`).
+When OM handles `PutObjectRetention` on a key with an active event hold, it clears `eventHold` under the key write lock
+and sets `retentionDate` to the later of the requested `RetainUntilDate` and the leader-stamped request time plus the event hold `rule`.
+The retention mode cannot be changed by this request, so the usual rules for shortening retention still apply.
+
+> _**Note**:
+> * Background & Root Cause: A prerequisite for enabling WORM (Write Once, Read Many) in AWS S3 is that Object Versioning must be enabled. Under S3 architecture, executing a Put on a locked object generates a new version without affecting the protected prior version; thus, S3 Object Lock primarily restricts Delete Object.
+> * Ozone Implementation Status: Because Ozone's versioning feature is still under development, to guarantee absolute immutability during the lock period, Ozone will directly block and reject all overwrite operations (such as any form of Put or overwrite) on locked objects.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional RetentionConfig defaultRetention = 25;
+ }
+
+ message RetentionConfig {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional Retention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message Retention {
+ required RetentionMode retentionMode = 1;
+ optional string retentionDate = 2;
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Ranger Access Control
+
+Ozone Object Lock enforces a dual-gate protection mechanism combining **Ranger policy-based authorization** and **underlying OM WORM state validation**:
+
+* **Standard Data Operations (Put / Delete)**: Even if a principal has valid Ranger `CREATE`, `WRITE` or `DELETE` access, any attempt to mutate, overwrite, or delete an object under an active Legal Hold or an unexpired Retention period is immediately rejected with `403 Access Denied` (`WORMProtectionException`).
+* **Fine-Grained S3 Action Matching**: Object Lock management integrates with the action-matching authorization framework introduced via STS. S3 Object Lock actions map 1:1 to AWS S3 action names. See [AWS STS Design for Ozone S3](ozone-sts.md) for details.
+
+---
+
+#### 1. Configuration & Component Updates
+
+To enable action-level authorization and onboard Object Lock actions into the Ozone & Ranger ecosystem, the following updates are required:
+
+**Prerequisite: Feature Flag Activation**:
+* Enable the Ranger action matcher condition in the Ozone configuration:
+ ```properties
+ ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+ ```
+* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+
+1. **UI & Metadata Registration**:
+ * Update `security-admin/src/main/webapp/react-webapp/src/utils/actionRequirements/ozone.json` in Apache Ranger:
+ Register the 7 new Object Lock actions along with their prerequisite primitive permissions (e.g., `READ`, `WRITE`) mapped by resource level (Bucket vs. Key). This enforces design-time validation in the Ranger React UI, preventing administrators from creating invalid policy items.
+
+2. **S3 Gateway & STS Mapping**:
+ * Update **`S3GActionIamMapper`**: Map incoming HTTP request context and sub-resources (e.g., `?legal-hold`, `?retention`, `?object-lock`) to the corresponding action string.
+ * Update **`IamSessionPolicyResolver`**: Ensure session-scoped policies (STS AssumeRole / federation tokens) recognize and evaluate the new Object Lock action strings.
+
+---
+
+#### 2. Action to Ranger Access Type Mapping
+
+Following the declarative contract in `ozone.json`, each S3 action requires a baseline primitive Ranger permission at the designated resource hierarchy:
+
+| S3 Action | Resource Level | Required Ranger Access Type | Description |
+|:---------------------------------------|:---------------|:----------------------------|:--------------------------------------------------------------------|
+| **`GetBucketObjectLockConfiguration`** | Bucket | `READ` | Retrieve default Object Lock settings on a bucket. |
+| **`PutBucketObjectLockConfiguration`** | Bucket | `WRITE` | Configure default retention mode and period on a bucket. |
+| **`GetObjectRetention`** | Key | `READ` | Read the retention mode and Retain-Until date of an object version. |
+| **`PutObjectRetention`** | Key | `WRITE` | Set or extend object retention mode and duration. |
+| **`GetObjectLegalHold`** | Key | `READ` | Query the current Legal Hold status (`ON` or `OFF`). |
+| **`PutObjectLegalHold`** | Key | `WRITE` | Toggle the Legal Hold state (`ON` or `OFF`). |
+| **`BypassGovernanceRetention`** | Key | `WRITE` and `DELETE` | Privileged entitlement to bypass retention in Governance Mode. |
+
+---
+
+#### 3. Value-Level Authorization & Ranger Condition Evaluator
+
+While API invocation is controlled by the S3 Action (e.g., granting `PutObjectLegalHold`), real-world governance often requires **role segregation based on request values** (e.g., Compliance Officers can toggle `ON`, but only external Auditors can toggle `OFF`; or preventing operators from setting `COMPLIANCE` mode).
+
+To support this without fragmenting AWS-compatible Action definitions, Ozone and Ranger introduce a payload-aware **String Matcher Condition Evaluator**:
+
+1. **Context Extraction in Ozone S3G**:
+ * During request interception, S3G inspects the request body/headers and injects target attributes into the `RangerAccessRequest.getContext()`:
+ * `s3:object-lock-legal-hold`: Extracted from the XML body (`ON` or `OFF`).
+ * `s3:object-lock-mode`: Extracted from the XML body (`GOVERNANCE` or `COMPLIANCE`).
+
+2. **Ranger Custom Condition Evaluator**:
+ * Implement a dedicated condition evaluator extending `RangerAbstractConditionEvaluator` (registered in `ranger-servicedef-ozone.json`):
+ * Evaluates `s3:object-lock-legal-hold` against configured policy criteria (`ON` vs. `OFF`).
+ * Evaluates `s3:object-lock-mode` against configured policy criteria (`GOVERNANCE` vs. `COMPLIANCE`).
+ * Enables fine-grained policies such as:
+ * *Policy A (Evidence Preservation)*: Allow `PutObjectLegalHold` **ONLY IF** `s3:object-lock-legal-hold == "ON"`.
+ * *Policy B (Safe Retention)*: Allow `PutObjectRetention` **ONLY IF** `s3:object-lock-mode == "GOVERNANCE"`.
+
+---
+
+#### 4. Retention Modes & Governance Bypass Enforcement
+
+* **Compliance Mode**:
+ * The strictest WORM protection tier.
+ * Once applied, the lock is immutable. **No role, administrator, or Ranger permission** (including `BypassGovernanceRetention`) can overwrite, shorten, or delete the object version until the retention period expires.
+* **Governance Mode & `BypassGovernanceRetention`**:
+ * Protects objects from accidental deletion by general users while retaining operational flexibility for authorized personnel.
+ * **Dual-Check Enforcement**:
+ 1. **Client Declaration**: The client must explicitly set the `bypassGovernanceRetention` flag on the request. S3 Gateway sets it from the `x-amz-bypass-governance-retention: true` header; native Ozone clients pass it as a boolean argument.
+ It is carried in `KeyArgs` and `DeleteKeyArgs`, so it is available to every operation that checks Object Lock (delete, batch delete, overwrite, multipart upload, `PutObjectRetention`).
+ Because Ozone does not support versioning yet, this flag is an Ozone-specific design: when Object Lock is enabled without versioning, a Governance-locked key can only be deleted or overwritten with the flag.
+ 2. **Ranger Action Entitlement**: Ozone checks whether the principal has the **`BypassGovernanceRetention`** action granted in Ranger for the target key resource.
+ 3. If either check fails, the request is aborted with `403 Access Denied`.
+
+
+> _**Warning**: Permanent Data Immutability in Compliance Mode_
+>
+> Unlike AWS S3, where objects in Compliance mode cannot be deleted by any user (including root) and will only be destroyed if the entire AWS account itself is removed, Apache Ozone has no concept of a root or tenant account.
+> Once applied, a Compliance lock cannot be overridden or deleted by any admin or Ranger policy until the retention period expires.
+
+### New Ozone APIs
+
+Surfacing objectLockEnabled, default retention settings, and active WORM protection via ozone shell will be addressed in a follow-up phase.
+
+**OzoneBucket**
+
+```java
+public void setRetentionConfig(RetentionArgs retentionArgs);
+public void setRetention(String keyName, ObjectRetention retention, boolean bypassGovernanceRetention);
+public void setLegalHold(String keyName, boolean hold);
+```
+
+### Supported S3 APIs
+
+To ensure seamless integration with existing S3 clients (e.g., AWS CLI, Boto3) and applications, the Ozone S3 Gateway (S3G) will translate and support standard AWS S3 APIs.
+In alignment with AWS S3, HeadObject and GetObject responses will be enriched with lock-related headers (`x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold`) when the caller possesses the required GetObjectRetention or GetObjectLegalHold permissions. Without the permission the headers are omitted and the request still succeeds, so the extra authorization never fails a plain read.
+
+**Bucket-Level APIs:**
+* `CreateBucket`: Supports enabling Object Lock at creation time via the x-amz-bucket-object-lock-enabled: true header.
+* `GetBucketObjectLockConfiguration`: Retrieves the Object Lock status and the default retention configuration (if any) for a specified OBS bucket.
+* `PutBucketObjectLockConfiguration`: Enables Object Lock and default retention on an existing bucket. In alignment with AWS S3, passing the `x-amz-bucket-object-lock-token` header is supported when enabling lock on existing buckets.
+
+**Object-Level APIs:**
+* `PutObject / CreateMultipartUpload`: Supports setting retention and legal hold directly during object creation via `x-amz-object-lock-mode`, `x-amz-object-lock-retain-until-date`, and `x-amz-object-lock-legal-hold` request headers.
+* `PutObjectRetention`: Places a retention configuration on an object, specifying the retention mode (COMPLIANCE or GOVERNANCE) and the retention duration (in days or years).
+* `GetObjectRetention`: Retrieves the current retention configuration applied to an object.
+* `PutObjectLegalHold`: Applies or removes a Legal Hold configuration to the specified object.
+* `GetObjectLegalHold`: Retrieves the current Legal Hold status of an object.
+
+**Impacted Standard APIs:**
+Standard data mutating APIs will intercept the Object Lock state and return the `403 Access Denied` (or specifically `WORMProtectionException`) if a modification is attempted on a locked object:
+* `PutObject` / `CopyObject` (Overwrites will be rejected)
+* `DeleteObject` / `DeleteObjects` (Deletions will be rejected)
+* `CreateMultipartUpload` / `CompleteMultipartUpload` / `UploadPart` (Uploads will be rejected if the object is locked)
+* `CopyObject` does not copy the Object Lock configuration of the source object (same as AWS S3 by default); the destination gets the bucket default retention or the `x-amz-object-lock-*` headers of the request.
+* The rejection uses a new OM result code for `WORMProtectionException`, which S3 Gateway maps to `403 AccessDenied`; `DeleteObjects` reports it as a per-key `AccessDenied` error.
+* Lifecycle expiration: the background lifecycle service deletes keys through the same `DeleteKeys` path, so locked keys are skipped by the delete check.
+
+> _**Note**:
+> * **Object Lock Immutability & Future Versioning Binding**:
+> - **Current State (No Versioning)**: Once Object Lock is enabled on a bucket (`objectLockEnabled = true`), it cannot be disabled.
+> - **AWS S3 Alignment & Future Evolution**: In standard AWS S3, Object Lock inherently requires and automatically enables Versioning upon bucket creation. While Ozone temporarily decouples them because Ozone does not support Versioning yet, future implementations will align with S3 semantics: enabling Object Lock will automatically enforce Versioning, and Versioning can never be suspended or disabled thereafter.
+
+### Impact on Write and Delete Flows
+
+**Impact on Put Object Flow**
+
+
+
+During the Create Key phase, the system executes a preliminary WORM check in `preExecute` to enforce a fail-fast mechanism:
+
+1. Linearizability does not need to be guaranteed at this stage.
+2. It avoids Raft consensus overhead, conserving system resources.
+
+During the Commit Key phase, the system performs final WORM validation in `validateAndUpdateCache`:
+
+1. It guarantees linearizability, ensuring only one write request can succeed simultaneously on a WORM-protected object.
+2. Keys that fail to commit remain in an Open Key state, and their associated metadata and content will be reclaimed periodically by system background cleanup.
+
+**Impact on Multipart Upload Flow**
+
+Multipart Upload adopts the same design logic as Put Object to ensure operational consistency:
+1. Similar to the creation key flow, WORM checks during the Initiate Multipart Upload and Put Part phases are executed in `preExecute` for fail-fast behavior (linearizability is not yet required, minimizing consensus overhead).
+2. Similar to the commit key flow, WORM check during the Complete Multipart Upload phase is executed in `validateAndUpdateCache` to guarantee linearizability for writes under locked states.
+
+**Impact on Delete Object Flow**
+
+All delete operations must perform WORM validation during the `validateAndUpdateCache` phase to preserve linearizability and prevent accidental deletion of protected data.
+
+### Volume Force Delete
+
+A volume cannot be deleted while it contains locked keys (`VOLUME_NOT_EMPTY`), and `ozone sh volume delete -r` deletes keys
+one by one from the client, so it is blocked by Object Lock as well. Ozone has no account concept, so unlike AWS
+(where closing the account destroys all data, including Compliance-locked objects), there would be no way to decommission
+a tenant or volume until every retention period expires. An admin-only command is provided for this case,
+with the volume as the unit of deletion, as the closest analogue to deleting an AWS account:
+
+```text
+ozone admin om volume force-delete <volume>
+```
+
+* **Disabled by default**: guarded by a new OM config (for example `ozone.om.object.lock.volume.force.delete.enabled`, default `false`).
+ Deployments that must never allow removal of Compliance-locked data keep it disabled.
+* **Authorization**: OM administrators only (`ozone.administrators`). It is not exposed through S3 Gateway and is not covered by `BypassGovernanceRetention`.
+* **Execution**: OM marks the volume as deleting and rejects any new request on it. A background service then purges buckets and keys in batches
+ (including open keys, multipart uploads and snapshots), because a single Ratis transaction cannot delete a large volume. WORM checks are skipped only for this purge.
+ Deletion is irreversible.
+* **Audit**: the request and the purge result are written to the OM audit log.
+* **Trusted boundary**: this is the only way to remove Compliance-locked data through the Ozone API. The data is protected from everyone except a cluster administrator who explicitly enabled and ran this command.
+
+## Performance
+
+Introducing WORM validation into critical execution paths (such as Create Key and Commit Key) incurs minor overhead.
+To minimize impact, this design employs a two-phase validation strategy: preliminary filtering in `preExecute` provides fail-fast pruning to prevent invalid requests from reaching the consensus layer, followed by atomic validation in `validateAndUpdateCache` for linearizability.
+The additional metadata lookups are expected to have a small impact on throughput and latency; this will be verified with benchmarks and metrics during implementation.
+
+## Security
+
+Centralized access control via Ranger ensures all Object Lock operations (such as Retention Policy configuration and Legal Hold management) are enforced under strict access permissions.
+
+By leveraging the **STS action-matching framework**, Ozone maps standard S3 API actions (e.g., `BypassGovernanceRetention`, `PutObjectLegalHold`) directly to standard Ranger primitive access types (e.g., `WRITE` and `READ`). This fine-grained, action-level authorization enforces the Principle of Least Privilege **without requiring custom dedicated access types**. It effectively prevents unauthorized tampering or removal of locks, ensuring robust compliance and data immutability.
+
+### Trusted Boundary and Compliance Scope
+
+While AWS S3 can guarantee absolute immutability across all levels because its infrastructure is entirely managed and abstracted from users, Apache Ozone is a self-managed, software-defined storage system. It is important to define the trusted boundary to understand the scope of data protection and compliance.
+
+**Trusted Boundary:**
+The trusted boundary for this Object Lock design is defined at the **Application API Level**.
+* All data and metadata operations routed through the Ozone Java API, S3 Gateway, RPC interfaces, and standard Ozone clients are strictly governed by the Object Lock validation (`validateAndUpdateCache`) and Ranger access policies.
+* Within this boundary, immutability is strictly enforced. Unless explicitly authorized with specific privileged permissions (e.g., s3:BypassGovernanceRetention), even cluster administrators using standard client tools cannot overwrite or delete locked objects until the retention period expires or the legal hold is removed.
+
+**Out of Scope (Infrastructure Level bypass):**
+Since Ozone runs on customer-managed infrastructure, operations executed with OS-level `root` privileges or physical access to the server nodes bypass the application-level API boundary. The following scenarios are not protected by Ozone Object Lock:
+* **Direct Database Tampering:** Modifying or deleting key metadata directly from the underlying RocksDB instances while the node is offline.
+* **Low-Level DR Tools:** Using disaster recovery tools like `ozone repair` to explicitly restore the OM's DB directory from an older Ratis snapshot (taken before the lock was applied) or running `skip-ratis-transaction` to skip retention state entries.
+* **Storage Level Deletion:** Directly deleting block files from the DataNode filesystems.
+* **System Clock Tampering**: Artificially manipulating or rolling back the system clock (NTP/OS time) on OM or DataNode hosts to prematurely expire object retention periods or evade time-based WORM validation.
+* **Ranger Super Admin Privilege Abuse**: Modifying, disabling, or overriding Ozone authorization policies via Apache Ranger admin credentials (or direct access to Ranger’s backend database), allowing privileged operators to alter bucket WORM flags or grant unrestricted deletion permissions outside normal enforcement workflows.
+
+**Compliance Recommendation:**
+To achieve strict regulatory compliance (e.g., SEC 17a-4 or FINRA WORM requirements) in an on-premise deployment, Ozone's Object Lock must be complemented with strict infrastructure security measures. This includes enforcing rigid OS-level RBAC/IAM, restricting SSH/root access to OM/DN nodes, and continuously forwarding Ozone audit logs to a tamper-evident, external SIEM system to detect manual offline interventions.
+
+## Compatibility
+
+* **Table Schema Changes**: Since all newly introduced fields in the Bucket Table (`objectLockEnabled`, `defaultRetention`) and Key Table (`retention`, `legalHold`) are defined as `optional` in Protobuf, they are wire-compatible.
+ To avoid RocksDB serialization problems if the cluster is rolled back after new fields were written, a new OM layout feature gates Object Lock: until the upgrade is finalized, OM rejects every Object Lock request (bucket configuration, retention, legal hold, and `x-amz-object-lock-*` headers on Put / Multipart Upload), so no Object Lock metadata is persisted before finalization.
+* **OM Version Leveling**: OM version leveling is introduced primarily because this feature adds new OM APIs (such as `setRetentionConfig` and `setLegalHold`). The new OM version allows clients to determine whether the target OM cluster supports executing these Object Lock APIs during rolling upgrades.
+
+## Alternatives
+
+### 1. Why Not Use Native ACLs for Object Lock Management?
+
+* **Proposal**:
+ Support Native ACLs alongside or instead of Ranger to configure and manage Object Lock states (e.g., using Ozone native ACLs or POSIX-like ACLs at the Volume/Bucket/Key level).
+
+* **Why Rejected**:
+ 1. **Conflicting Security & Authorization Philosophies**:
+ Native ACLs in Ozone are primarily designed for granular, resource-centric permissions (often granting permissions directly to specific objects/buckets or inherited via path hierarchies). In contrast, Object Lock is an enterprise compliance feature that requires strict, role-based access governance (RBAC/ABAC). Compliance controls like WORM and Legal Holds are inherently tied to compliance officers, security auditors, or legal teams—not individual data owners.
+ 2. **Risk of Decentralized & Inconsistent Governance**:
+ Native ACLs allow object creators or bucket owners to adjust permissions. If lock controls were exposed through Native ACLs, standard bucket administrators or key owners could modify or override lock-related ACLs locally. Centralizing lock authorization exclusively in **Apache Ranger** ensures that compliance policies are managed from a unified pane of glass, preventing local policy drift or accidental misconfigurations.
+ 3. **Operational Complexity**:
+ Supporting dual authorization models (Ranger + Native ACLs) for critical compliance features introduces unnecessary complexity, potential security loopholes, and ambiguous resolution hierarchies during audits. Excluding Native ACLs ensures clean boundary enforcement.
+
+## Plan
+
+The implementation of S3 Object Lock in Apache Ozone is divided into six structured phases. This ensures incremental delivery, clean service boundaries, and testability at each stage.
+
+### Phase 1: Metadata Infrastructure & Schema Definitions
Review Comment:
Thanks for the suggestion. Updated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "Gargi-jais11 (via GitHub)" <gi...@apache.org>.
Gargi-jais11 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4356207718
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -0,0 +1,597 @@
+---
+title: S3 Object Lock
+summary: Design to support S3 object lock.
+date: 2026-08-18
+jira: HDDS-15945
+status: draft
+author: Chung En Lee
+---
+<!--
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License. See accompanying LICENSE file.
+-->
+
+# S3 Object Lock Design Doc
+
+## Summary
+
+This design adds S3 Object Lock to OBS buckets, so keys can be protected from deletion and overwrite.
+Authorization uses Ozone native ACLs or Apache Ranger.
+
+## Problem statement
+
+Data stored for security or compliance must be protected from accidental or malicious deletion and overwrite. Object Lock provides this protection.
+
+The design supports both native ACLs and Ranger. Native ACLs serve deployments without Ranger, and let the core logic be built and tested without the Ranger integration.
+To ship the core feature sooner, the first release excludes versioning, Legacy buckets and FSO buckets.
+
+**Goal:**
+
+* Object Lock on OBS buckets, with authorization through native ACLs or Ranger.
+* Single-version keys only.
+
+## Non-Goal
+
+* Versioning: locks across multiple object versions (versioning is still under development).
+* Legacy and FSO buckets. FSO needs its own decisions on:
+ * Lock granularity: files and directories are in different tables (FileTable / DirectoryTable), so locks could apply to files, directories or both.
+ * Rename and move: they change the namespace without changing the data, so they must either be blocked on locked paths or keep the lock with the same objectID.
+* Value-level Ranger authorization, for example allowing a legal hold to be set `ON` but not `OFF`.
+
+## Differences from AWS
+
+* Single version: locks apply to the current key, not to object versions.
+* Overwrites are rejected: in AWS S3, Put, Copy or MPU on a locked object writes a new version. In Ozone it would replace the locked key,
+ so Ozone rejects Put, Copy, MPU and rename (native clients only) on locked keys with `403 Access Denied` (`WORMProtectionException`).
+* Governance bypass on overwrite and rename: a Governance-locked key can only be overwritten or renamed with the `bypassGovernanceRetention` flag and the bypass permission.
+ AWS S3 does not need this because it writes a new version (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+* Authorization: Ozone has no root account.
+ * Native ACLs: changing locks and bypassing Governance need the volume or bucket owner; setting or changing a Compliance retention needs an Ozone administrator (see [Native ACLs](#1-native-acls)).
+ * Ranger: action-level permissions need `enableActionMatcherInPoliciesCondition`, which is `false` by default (see [Ranger](#2-ranger)).
+ Object Lock uses the same access types as data (`READ` / `WRITE`), so without `action-matches`, anyone who can write a key can also change its locks.
+ * No API can remove a Compliance retention, in either model. Only the offline `ozone repair` command can (see [Offline Compliance Retention Repair](#offline-compliance-retention-repair)).
+
+## Technical Description
+
+### Terminology
+
+**Legal Hold**
+
+* Locks a key with no end date, until someone allowed to change the legal hold removes it
+ (the owner with native ACLs; `WRITE` with Ranger, limited to `PutObjectLegalHold` when `action-matches` is used).
+* Used for litigation, investigations or audits, when it is not known how long the data must be kept.
+* Has no mode and no date, and is independent of retention: a key can have a legal hold, a retention, or both, and stays locked until both are gone.
+ A legal hold can be set on a key without a retention.
+* Blocks: Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients).
+* Allows: Get / Head, Get / Put Object Tagging, Get Legal Hold, Put Legal Hold (with permission).
+
+**Retention**
+
+* Protects a key until a `RetainUntilDate`. The date is either fixed (set directly, or from days / years) or comes from an event hold.
+* Modes:
+ * Compliance: no one, including administrators, can remove it, shorten it or overwrite the key through the Ozone API before it expires. Only the offline `ozone repair` command can change it.
+ * Governance: blocks normal users, but users with the bypass permission can bypass it.
+* Event hold (AWS S3 *variable retention*):
+ * Used when the retention depends on an external event, such as a contract ending, an employee leaving or a loan closing.
+ * While the event has not happened (`EventHold=ON`), the key is locked with no end date. When it happens, the hold is released (`EventHold=OFF`)
+ and the duration (`EventHoldDuration`, in days or years) becomes a fixed `RetainUntilDate`.
+ * See [Event Hold Trigger Workflow](#event-hold-trigger-workflow) for the rules.
+* Blocks (Governance allows them with `bypassGovernanceRetention`): Put / Copy (overwrite), Delete, Multipart Upload (initiate / complete), Rename (native clients),
+ shortening or removing the retention (never allowed for Compliance).
+* Allows: Get / Head, Get / Put Object Tagging, extending the retention.
+
+#### Event Hold Trigger Workflow
+
+```text
+object created
+ |
+event hold applied (locked with no end date)
+ |
+business event occurs
+ |
+hold released
+ |
+RetainUntilDate is calculated
+ |
+locked until RetainUntilDate
+```
+
+Event holds use `PutObjectRetention`, as in AWS S3, so no new API or Ranger action is needed (native clients use `OzoneBucket#setRetention`).
+OM handles the request under the key write lock and uses the leader-stamped request time as now.
+
+The key stores `retentionDate` and the event hold (status and duration):
+* The duration comes from the request, or from the bucket default when the key is created. Later changes to the bucket default do not affect existing keys.
+* If the bucket default has both `rule` and `eventHold`, a new key gets `retentionDate` = creation time + `rule` and an active event hold.
+ With only `eventHold`, `retentionDate` is unset until the hold is released.
+* While the hold is `ON`, the effective `RetainUntilDate` is the later of `retentionDate` and now + duration, so it moves forward over time.
+ `GetObjectRetention`, `HeadObject` and `GetObject` return this date.
+
+Below, "earlier" and "later" compare the requested `RetainUntilDate` with the effective date. Only `GOVERNANCE` with the bypass can make it earlier; `COMPLIANCE` never can.
+
+1. **Request format**: checked before the lock and the bypass; a request that breaks a rule gets `400 Bad Request`.
+ * The request has a mode, and `RetainUntilDate`, `EventHold` or both.
+ The only exception is an empty retention, which removes the retention: it needs the bypass for `GOVERNANCE` (`403` without it) and is never allowed for `COMPLIANCE`.
+ * `RetainUntilDate` is in the future.
+ * `EventHold=ON` has exactly one `EventHoldDuration` (positive, at most 36,500 days or 100 years). A `RetainUntilDate` in the same request is at least now + duration.
+ * `EventHold=OFF` has no `EventHoldDuration`, and `EventHoldDuration` needs `EventHold=ON`.
+ * The same applies to the `x-amz-object-lock-*` headers at creation: the mode header needs the retain-until-date header or the event hold headers, and they need the mode header.
+2. **`EventHold=ON`** sets or updates the hold.
+ * Without the bypass, `RetainUntilDate` can only be later (`403` otherwise). The duration can change;
+ when it gets shorter, OM first saves the current effective date in `retentionDate`, so the date never moves earlier, even with the bypass.
+ * With the bypass (`GOVERNANCE` only), `RetainUntilDate` can be earlier, but still at least now + the new duration (rule 1).
+3. **`EventHold=OFF`** releases the hold. It is the only way to release it.
+ * Without `RetainUntilDate`: `retentionDate` = the later of `retentionDate` and now + duration.
+ * With `RetainUntilDate`: that date is used. An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`.
+ * On a key without an active hold: no change if the key has a future `retentionDate`; `400` if neither the key nor the request has a future date.
+4. **No `EventHold`** keeps the hold as it is and only changes `RetainUntilDate`.
+ * A later date is accepted, also while the hold is `ON`.
+ * An earlier date needs the bypass (`GOVERNANCE` only), otherwise `403`. While the hold is `ON`, it gets `400` even with the bypass: the request must use rule 2 or 3.
+
+Releasing the hold keeps the mode. A different mode in the request is a mode change (see [Retention Updates](#retention-updates)).
+
+> _**Note**_: AWS S3 requires versioning for Object Lock, so a Put on a locked object only adds a new version and Object Lock mainly blocks deletes.
+> Ozone has no versioning yet, so it rejects every overwrite of a locked key.
+
+### Table Changes
+
+**Bucket Table**
+
+Two new fields: objectLockEnabled & defaultRetention.
+
+```protobuf
+ message BucketInfo {
+ // ... existing fields
+ optional bool objectLockEnabled = 24 [default = false];
+ optional BucketRetention defaultRetention = 25;
+ }
+
+ message BucketRetention {
+ required RetentionMode retentionMode = 1;
+ optional Rule rule = 2;
+ optional EventHold eventHold = 3;
+ }
+
+ message EventHold {
+ required bool enabled = 1;
+ required Rule rule = 2;
+ }
+
+ message Rule {
+ required TimeUnit timeUnit = 1;
+ required uint64 duration = 2;
+ }
+
+ enum TimeUnit {
+ DAYS = 1;
+ YEARS = 2;
+ }
+
+ enum RetentionMode {
+ GOVERNANCE = 1;
+ COMPLIANCE = 2;
+ }
+```
+
+
+
+**Key Table**
+
+Two new fields: retention & legalHold.
+
+```protobuf
+message KeyInfo {
+ // ... existing fields
+ optional KeyRetention retention = 23;
+ optional bool legalHold = 24 [default = false];
+}
+
+message KeyRetention {
+ required RetentionMode retentionMode = 1;
+ optional uint64 retentionDate = 2; // epoch milliseconds; the minimum date while the event hold is ON
+ optional EventHold eventHold = 3;
+}
+```
+
+
+
+### Access Control
+
+A request on a locked key must pass two checks: authorization (native ACLs or Ranger) and the WORM check in OM.
+Even with `CREATE`, `WRITE` or `DELETE`, overwriting or deleting a locked key is rejected with `403 Access Denied` (`WORMProtectionException`),
+unless the Governance bypass applies (see [Retention Modes and Governance Bypass](#3-retention-modes-and-governance-bypass)).
+
+The cluster uses one of two authorization models:
+* [Native ACLs](#1-native-acls) have no per-action permissions, so changing locks needs the owner, and setting or changing a Compliance retention needs an Ozone administrator.
+* [Ranger](#2-ranger) checks access types, and `action-matches` can limit a policy item to specific S3 actions. Use Ranger when each Object Lock operation needs its own permission.
+
+In both models, keys that only get the bucket default retention need no extra check, because the default was authorized when it was set.
+Native clients pass the same settings and the `bypassGovernanceRetention` flag as request arguments, and the same rules apply.
+
+---
+
+#### 1. Native ACLs
+
+| Operation | Resource | Required |
+|:----------------------------------------------------------------------------------|:-------------|:--------------------|
+| `GetBucketObjectLockConfiguration` | Bucket | `READ` |
+| `GetObjectRetention`, `GetObjectLegalHold` | Key | `READ` |
+| `PutBucketObjectLockConfiguration` (no default, or `GOVERNANCE`) | Bucket | Owner |
+| `PutObjectRetention` on a `GOVERNANCE` key or a key without retention (setting `GOVERNANCE`), `PutObjectLegalHold` | Key | Owner |
+| Governance bypass (including shortening or removing a `GOVERNANCE` retention) | Key | Owner |
+| Release the event hold of a `COMPLIANCE` key (`EventHold=OFF` without `RetainUntilDate`) | Key | Owner |
+| Set or change `COMPLIANCE`: on a key (including extending it), `GOVERNANCE` → `COMPLIANCE`, or as the bucket default (also changing a `COMPLIANCE` default) | Bucket / Key | Ozone administrator |
+
+* **Owner**: the owner of the volume or bucket, the same owner that `OzoneNativeAuthorizer` already lets skip ACL checks. Ozone administrators can do everything an owner can.
+ Locks use the owner instead of an ACL type like `WRITE_ACL`, because any ACL type can be granted to other users.
+* **Ozone administrator**: a user in `ozone.administrators` or a group in `ozone.administrators.groups` (if neither is set, the user that starts OM).
+ Read-only and S3 administrators are not included. The role is cluster wide; use Ranger for finer control.
+* Reading lock information only needs `READ`, because `HeadObject` / `GetObject` already return it in the `x-amz-object-lock-*` headers.
+* Every change to a Compliance retention needs an administrator, because no API can undo it: setting it, and also extending it (a later date or a longer event hold duration),
+ which could otherwise lock a key for up to 100 years. This is stricter than AWS S3, where `PutObjectRetention` is enough.
+* The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the stored duration, which an administrator already approved, into a date,
+ so the owner can do it when the business event happens. Releasing with a `RetainUntilDate` needs an administrator.
+* Keys that get a Compliance bucket default need no administrator check. To store Compliance data, an administrator sets `COMPLIANCE` as the bucket default once,
+ and applications upload as normal users.
+
+##### Extra Checks for Object Lock Headers
+
+Requests that carry Object Lock headers also need the owner or an administrator, checked in the same request after the normal ACL check:
+
+| Request | Header or condition | Extra check |
+|:--------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------|:---------------------------------------------------------|
+| `CreateBucket` | `x-amz-bucket-object-lock-enabled: true` | Owner |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-mode`, `-retain-until-date`, `-event-hold`, `-event-hold-duration-days` / `-years` | Owner (`GOVERNANCE`), Ozone administrator (`COMPLIANCE`) |
+| `PutObject`, `CopyObject` (destination), `CreateMultipartUpload` | `x-amz-object-lock-legal-hold` | Owner |
+| `DeleteObject`, `DeleteObjects`, overwrite, rename (native clients) | `bypassGovernanceRetention` on a key under Governance retention | Owner |
+
+If the check fails, the request fails (`DeleteObjects` reports a per-key `AccessDenied`). `GetObject` / `HeadObject` need no extra check and always return the lock headers.
+So a user who can only write data cannot set a lock by adding a header.
+
+---
+
+#### 2. Ranger
+
+> _**Important**_: action-level permissions need the Ranger action matcher, which is `false` by default.
+> Without it, every principal with `WRITE` on a key can also change its locks, including setting a Compliance retention.
+> ```properties
+> ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
+> ```
+
+* Object Lock uses the same access types as data: `READ` to read lock information, `WRITE` to change it, and `WRITE` and `DELETE` for the Governance bypass.
+ They apply to every client: S3 Gateway, ofs / o3fs and the Ozone CLI.
+* Ranger has no administrator role, so setting Compliance needs no special permission: `PutObjectRetention` (`WRITE`) is enough, unlike native ACLs, where it needs an Ozone administrator.
Review Comment:
Thanks @chungen0126, the reasoning makes sense. Agreed that a cluster-wide admin is too coarse and that per bucket delegation with `action-matches` is the main reason to use Ranger here, so I'm fine keeping the AWS behavior.
Since the offline repair is now the recovery path for a Compliance retention set by mistake, So this should highlight in the doc about the repair section (line 464) and describe how to run it with OM HA?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on PR #11053:
URL: https://github.com/apache/ozone/pull/11053#issuecomment-6282535132
Thanks @ChenSammi @devmadhuu @fmorg-git @Gargi-jais11 for the review. I update the latest version, please take a look if you are available.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org
Re: [PR] HDDS-15945. [OEP] Support S3 WORM (Object Lock) [ozone]
Posted by "chungen0126 (via GitHub)" <gi...@apache.org>.
chungen0126 commented on code in PR #11053:
URL: https://github.com/apache/ozone/pull/11053#discussion_r4311746581
##########
hadoop-hdds/docs/content/design/s3-object-lock.md:
##########
@@ -203,7 +241,18 @@ To enable action-level authorization and onboard Object Lock actions into the Oz
```properties
ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition=true
```
-* When enabled, Ozone S3 Gateway (S3G) intercepts the high-level S3 API operation and populates the S3 action name into the `RangerAccessRequest`, allowing Ranger's `RangerActionMatcher` to evaluate policy conditions against incoming requests.
+* When enabled, the S3 action name in the `RangerAccessRequest` is evaluated by Ranger's `RangerActionMatcher` against policy conditions.
+ Without this flag, Ranger ignores the action and only checks the primitive access type (for example, plain `WRITE` for `PutObjectLegalHold`),
+ so this flag is a prerequisite for using Object Lock with Ranger.
+
+**OM Determines the Object Lock Action**:
+* Today OM forwards `S3Authentication.s3Action` from S3 Gateway to Ranger as is, and only when `ozone.s3g.sts.http.enabled=true` (default `false`).
+ For Object Lock, OM must not depend on that string, because a wrong mapping in S3 Gateway, a native client that sends no action,
+ or STS being disabled would all reduce the check to plain `WRITE`.
+* Instead, each Object Lock OM request sets the S3 action in the `RequestContext` itself, based on the request type:
+ `OMKeySetLegalHoldRequest` always checks `PutObjectLegalHold`, `OMKeySetRetentionRequest` checks `PutObjectRetention`, the bucket request checks `PutBucketObjectLockConfiguration`,
+ and any request with `bypassGovernanceRetention = true` additionally checks `BypassGovernanceRetention`.
+* As a result, Object Lock with Ranger does not require STS to be enabled, and S3 Gateway, native clients and STS sessions are checked the same way.
Review Comment:
@ChenSammi @fmorg-git Updated the doc:
- Native ACLs: following @ChenSammi's proposal fully, every Compliance change needs an Ozone administrator, including extending it, because no API can undo that either. The one exception is releasing the event hold of a Compliance key without a `RetainUntilDate`: it only turns the duration an administrator already approved into a date, so the owner can do it when the business event happens. Keys that inherit a Compliance bucket default need no extra check, so applications can upload Compliance data as normal users.
- Ranger: reading lock information needs `READ`, and changing locks needs `WRITE` (plus `DELETE` for the bypass), the same access types as for data, instead of `READ_ACL` / `WRITE_ACL`. This matches what @fmorg-git described earlier (for example `PutObjectLegalHold` with `write`). Ranger has no administrator role, so setting Compliance only needs `PutObjectRetention`, as in AWS S3, and `action-matches` is what separates lock management from data writes.
- `ozone repair om object-lock` is for both models: no API can relax a Compliance retention, so one set by mistake can only be removed offline.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org