You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "thswlsqls (via GitHub)" <gi...@apache.org> on 2026/09/27 03:51:13 UTC

[PR] [filesystem][s3] Mirror s3.session-token to fs.s3a.session.token [paimon]

thswlsqls opened a new pull request, #10228:
URL: https://github.com/apache/paimon/pull/10228

   
   ### Purpose
   
   fix #10227
   
   - `S3FileIO` mirrors hyphenated `access-key`/`secret-key` to the dotted S3A keys, but not the session token.
   - `s3.session-token` became `fs.s3a.session-token`, which S3A never reads, so temporary STS credentials passed via Flink/Spark/Hive catalog options were signed without the token (403).
   - Adds `{"fs.s3a.session-token", "fs.s3a.session.token"}` to `MIRRORED_CONFIG_KEYS`; precedence matches the existing mirrors. Dotted `s3.session.token` is unaffected.
   - Aligns with PyPaimon, which accepts `s3.session-token` (#7712).
   - Documents `s3.session-token` in the S3 section of `filesystems.mdx`.
   
   ### Tests
   
   - Added offline `S3FileIOConfigTest`: `testHyphenSessionTokenIsMirrored` (conf key plus resolved `AwsSessionCredentials`), `testDotSessionTokenIsKept`.
   - Without the fix, `testHyphenSessionTokenIsMirrored` fails (`fs.s3a.session.token` is null).
   - `mvn -pl paimon-filesystems/paimon-s3-impl clean install` green on JDK 11 (`S3FileIOConfigTest` 2/2, spotless/checkstyle); Docker-based MinIO tests not run locally.
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [filesystem][s3] Mirror s3.session-token to fs.s3a.session.token [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on PR #10228:
URL: https://github.com/apache/paimon/pull/10228#issuecomment-5944677154

   Requirement fit: SUPPORTED. Implementation: CLEAN at `e354f8a5c6`.
   
   The temporary-credential fix has caller-visible value, and I found no introduced code defect. Both new configuration tests pass locally with normal Maven checks. I also exercised real S3A/SDK HTTP reads against a local endpoint that rejects requests missing the session token: all three hyphenated prefixes (`s3.`, `s3a.`, `fs.s3a.`), plus the existing dotted option, return the expected content and include `x-amz-security-token` in the signed request. Local Docker was unavailable, so this probe does not claim live AWS/MinIO STS verification.
   
   One production-validation gap remains: [Flink 1 / Common](https://github.com/apache/paimon/actions/runs/36292610298/job/108545483717) was cancelled during “Build dependencies and run tests”, leaving the aggregate CI result failed. The last log reports completed passing tests, then cancellation; it does not establish completion of that suite. The Core/integrations JDK 8 and 11 checks are green. Please obtain a completed green Flink 1 check before merging.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [filesystem][s3] Mirror s3.session-token to fs.s3a.session.token [paimon]

Posted by "thswlsqls (via GitHub)" <gi...@apache.org>.
thswlsqls commented on PR #10228:
URL: https://github.com/apache/paimon/pull/10228#issuecomment-5946309621

   @JingsongLi Thanks for the review. Flink 1 / Common now completes green on `84bc789a7`.
   
   The earlier run hit the suite's 100-minute timeout after `SinkSavepointITCase` stopped producing output, unrelated to the S3 option change. I merged the latest master to retrigger CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [filesystem][s3] Mirror s3.session-token to fs.s3a.session.token [paimon]

Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi merged PR #10228:
URL: https://github.com/apache/paimon/pull/10228


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org