[jira] [Commented] (HDDS-14893) Support single-node local S3 development

["Wei-Chiu Chuang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDDS-14893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18080443#comment-18080443 ] 

Wei-Chiu Chuang commented on HDDS-14893:
----------------------------------------

The trino community could benefit from this feature. Their test framework centers around docker containers and having a readily usable single node docker contianer greatly improves test quality for them.

> Support single-node local S3 development
> ----------------------------------------
>
>                 Key: HDDS-14893
>                 URL: https://issues.apache.org/jira/browse/HDDS-14893
>             Project: Apache Ozone
>          Issue Type: New Feature
>          Components: Tools
>            Reporter: Chu Cheng Li
>            Assignee: Chu Cheng Li
>            Priority: Major
>
> Apache Ozone is S3-compatible, but local setup is still much heavier than tools like MinIO. This makes it harder to use Ozone in the two most common lightweight adoption paths:
> 1. devcontainers / application development, where developers only need a local S3 endpoint to verify SDK or application behavior
> 2. example stacks such as Iceberg + Spark + Trino, where users need a simple local object store for demos and tutorials
> Add a local launcher, `ozone local run`, that starts a small single-node Ozone cluster with sensible defaults for local development.
> The command should:
> - start SCM, OM, one or more DNs, and optional S3G in one local runtime
> - support persistent local data directories
> - support fixed ports and bind/advertised host overrides
> - be configurable directly from CLI flags and `OZONE_LOCAL_*` environment variables so it works naturally with Docker Compose and devcontainers
> - print a clear startup summary including the S3 endpoint and suggested AWS client environment variables
> - make insecure local S3 usage easy, without requiring a separate `ozone s3 getsecret` bootstrap step
> For local insecure clusters, the launcher should present a stable recommended credential pair for examples and documentation, while also making it clear that insecure local S3 does not require real secret provisioning.
> Initial use cases:
> - `docker compose up` for local app development against S3
> - local Iceberg / Spark / Trino demos using Ozone S3G
> - smoke testing with AWS CLI and standard SDKs
> Acceptance criteria:
> - `ozone local run` can boot a usable local cluster with S3G enabled
> - Docker Compose can configure it via `OZONE_LOCAL_*` environment variables
> - AWS CLI can create a bucket, upload an object, and read it back from the local S3 endpoint
> - startup output clearly shows endpoint and client settings
> - invalid env/CLI values are reported with clear parse errors
> Non-goals for the first version:
> - HA/local multi-service production deployment semantics
> - external YAML profile files
> - secure-cluster credential/bootstrap workflow beyond the normal existing Ozone security model



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org