[I] datalake_fdw: storage I/O over object storage (S3) [cloudberry]

["MisterRaindrop (via GitHub)" <gi...@apache.org>]

MisterRaindrop opened a new issue, #2009:
URL: https://github.com/apache/cloudberry/issues/2009

   Part of #2008. Letters (A, B0–B7, C, D, E) are the PRs listed there; this is **A**.
   
   ### Scope
   - An S3 backend (AWS SDK for C++) in `backend_registry`, configured from the volume server and user-mapping options the validator already accepts (endpoint, region, path style, credentials, session token).
   - Arrow `RandomAccessFile` / `OutputStream` adapters over `common/file_system_wrapper.h`; `parquet_read.cpp` and `parquet_write.cpp` are the only format files that change.
   - The writer's rule carried over: an object is created once, never overwritten, and a failed writer removes exactly that object.
   - A `storage_s3` smoke category against MinIO, and the CI job that starts MinIO (the `CATEGORY_SERVICES` hook in `run_smoke_tests.sh`).
   
   ### Out of scope
   HDFS (libhdfs3 is unmaintained; the wrapper leaves room for a second backend). Iceberg path layout (C).
   
   ### Depends on
   Nothing. Starts first.
   
   ### Acceptance
   - The #1951 type set round-trips through `s3://` against MinIO in CI.
   - A write aborted after the first row group leaves no object; an existing key is refused.
   - Credentials never appear in logs, errors or `EXPLAIN`.
   - `nm -D` exports only the PostgreSQL entry points; Arrow 9 and 17 build.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org

Re: [I] datalake_fdw: storage I/O over object storage (S3) [cloudberry]

["MisterRaindrop (via GitHub)" <gi...@apache.org>]

MisterRaindrop closed issue #2009: datalake_fdw: storage I/O over object storage (S3)
URL: https://github.com/apache/cloudberry/issues/2009


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cloudberry.apache.org
For additional commands, e-mail: commits-help@cloudberry.apache.org