You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Chung-En Lee (Jira)" <ji...@apache.org> on 2026/09/23 13:41:00 UTC
[jira] [Updated] (HDDS-16531) Support direct S3 upload via Presigned PUT to DataNodes
[ https://issues.apache.org/jira/browse/HDDS-16531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chung-En Lee updated HDDS-16531:
--------------------------------
Description:
h5. Motivation
In current Ozone deployments, all S3 traffic (both metadata and payload) traverses the S3 Gateway (S3G). S3G acts as an active reverse-proxy, translating HTTP streams into internal Ozone client RPC/gRPC/Ratis calls.
This architecture has major drawbacks for write-intensive workloads:
# {*}Network Double-Hop{*}: Client -> S3G -> DataNodes, making S3G instances a massive network and CPU bottleneck.
# {*}Unnecessary Proxy Overhead{*}: S3G must buffer and manage streaming chunks, increasing latency and memory pressure.
# {*}Deviation from Modern Object Storage Pattern{*}: Standard S3 Presigned PUT is intended to offload data transmission directly to storage nodes after obtaining authorization from the control plane.
h5. Proposed Architecture: Decoupled S3 Data Path
We propose decoupling S3G into a pure *Control Plane* (Metadata & Signing) and leveraging *DataNodes (or co-located Data Workers)* as the {*}Data Plane{*}:
{code:java}
1. Request Presigned URL
Client -------------------> S3G / OM (Control Plane)
<------------------- Returns Presigned URL (pointing to target DN)
2. Direct Payload Upload (HTTP PUT)
Client -----------------------------------------------------> Leader DataNode (Data Plane)
│
├──> Replicate via Ratis Pipeline
│ (Follower DNs)
└──> Ack / Commit {code}
h5. Goal of this JIRA
Start an architectural discussion and gather feedback on:
* Feasibility of embedding an HTTP receiver into DataNode.
* Security model for validating presigned parameters on storage nodes.
* Preparation of a formal design doc for community review.
was:
h5. Motivation
In current Ozone deployments, all S3 traffic (both metadata and payload) traverses the S3 Gateway (S3G). S3G acts as an active reverse-proxy, translating HTTP streams into internal Ozone client RPC/gRPC/Ratis calls.
This architecture has major drawbacks for write-intensive workloads:
# {*}Network Double-Hop{*}: Client -> S3G -> DataNodes, making S3G instances a massive network and CPU bottleneck.
# {*}Unnecessary Proxy Overhead{*}: S3G must buffer and manage streaming chunks, increasing latency and memory pressure.
# {*}Deviation from Modern Object Storage Pattern{*}: Standard S3 Presigned PUT is intended to offload data transmission directly to storage nodes after obtaining authorization from the control plane.
h5. Proposed Architecture: Decoupled S3 Data Path
We propose decoupling S3G into a pure *Control Plane* (Metadata & Signing) and leveraging *DataNodes (or co-located Data Workers)* as the {*}Data Plane{*}:
{code:java}
1. Request Presigned URL
Client -------------------> S3G / OM (Control Plane)
<------------------- Returns Presigned URL (pointing to target DN)
2. Direct Payload Upload (HTTP PUT)
Client -----------------------------------------------------> Leader DataNode (Data Plane)
│
├──> Replicate via Ratis Pipeline
│ (Follower DNs)
└──> Ack / Commit {code}
h5. Key Technical Considerations
# {*}DataNode HTTP Ingress{*}:
** Introduce a lightweight HTTP/Netty ingress service on DataNodes (disabled by default or on a dedicated port).
** It accepts HTTP PUT requests carrying pre-authenticated authorization context.
# {*}Authentication & Token Encapsulation{*}:
** How S3G encodes Ozone credentials: S3G validates AWS SigV4 / credentials during URL generation, requests block/container allocation from OM, and embeds an Ozone Block Token & pipeline metadata into the Presigned URL.
** DN validates the embedded token before accepting the payload.
# {*}Ratis Bridge on Leader DataNode{*}:
** The Leader DN acts as the stream head, converting incoming HTTP payload directly into Ratis log entries / zero-copy chunk writes across the allocated pipeline.
# {*}Lifecycle & Multipart Upload{*}:
** For single PUT: Automatic OM commit upon stream completion, or explicit finalization.
** For Multipart Upload: Individual parts uploaded directly to DNs; {{CompleteMultipartUpload}} processed by S3G / OM as a metadata-only transaction.
h5. Goal of this JIRA
Start an architectural discussion and gather feedback on:
* Feasibility of embedding an HTTP receiver into DataNode vs. a standalone edge sidecar.
* Security model for validating presigned parameters on storage nodes.
* Preparation of a formal design doc for community review.
> Support direct S3 upload via Presigned PUT to DataNodes
> -------------------------------------------------------
>
> Key: HDDS-16531
> URL: https://issues.apache.org/jira/browse/HDDS-16531
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Chung-En Lee
> Priority: Major
>
> h5. Motivation
> In current Ozone deployments, all S3 traffic (both metadata and payload) traverses the S3 Gateway (S3G). S3G acts as an active reverse-proxy, translating HTTP streams into internal Ozone client RPC/gRPC/Ratis calls.
> This architecture has major drawbacks for write-intensive workloads:
> # {*}Network Double-Hop{*}: Client -> S3G -> DataNodes, making S3G instances a massive network and CPU bottleneck.
> # {*}Unnecessary Proxy Overhead{*}: S3G must buffer and manage streaming chunks, increasing latency and memory pressure.
> # {*}Deviation from Modern Object Storage Pattern{*}: Standard S3 Presigned PUT is intended to offload data transmission directly to storage nodes after obtaining authorization from the control plane.
> h5. Proposed Architecture: Decoupled S3 Data Path
> We propose decoupling S3G into a pure *Control Plane* (Metadata & Signing) and leveraging *DataNodes (or co-located Data Workers)* as the {*}Data Plane{*}:
>
> {code:java}
> 1. Request Presigned URL
> Client -------------------> S3G / OM (Control Plane)
> <------------------- Returns Presigned URL (pointing to target DN)
>
> 2. Direct Payload Upload (HTTP PUT)
> Client -----------------------------------------------------> Leader DataNode (Data Plane)
> │
> ├──> Replicate via Ratis Pipeline
> │ (Follower DNs)
> └──> Ack / Commit {code}
> h5. Goal of this JIRA
> Start an architectural discussion and gather feedback on:
> * Feasibility of embedding an HTTP receiver into DataNode.
> * Security model for validating presigned parameters on storage nodes.
> * Preparation of a formal design doc for community review.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org