You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@fluss.apache.org by "litiliu (via GitHub)" <gi...@apache.org> on 2026/06/25 08:55:40 UTC

[I] [client][filesystem] Reject credential-bearing client.fs.* options [fluss]

litiliu opened a new issue, #3524:
URL: https://github.com/apache/fluss/issues/3524

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.
   
   ### Fluss version
   
   1.0-snapshot
   
   ### Please describe the bug
   
   PR #938 introduced `client.fs.*` forwarding so Fluss clients can pass client-side filesystem options when they need to access remote files, such as KV snapshots or remote log segments.
   
   The intended scope appears to be necessary client-side filesystem runtime options, not filesystem credential configuration. In the PR discussion, exposing credential-related filesystem options such as `*.accessKeySecret`, `*.roleArn`, and `*.credentials.provider` was explicitly called out as dangerous, and the PR was updated to forward only the `client.fs.*` namespace.
   
   However, the current implementation still forwards all `client.fs.*` options to `FileSystem.initialize` after stripping the `client.fs.` prefix. This means credential-bearing options can be provided by the client and can affect client-side filesystem initialization.
   
   For S3, this can bypass the intended server-issued temporary token flow. Fluss clients are expected to obtain filesystem security tokens from Fluss servers through `getFileSystemSecurityToken` and use `DynamicTemporaryAWSCredentialsProvider` locally. Long-lived credentials, AssumeRole, and STS token generation should be managed by Fluss servers.
   
   For example, a client can configure:
   
   ```text
   client.fs.s3a.access.key
   client.fs.s3a.secret.key
   client.fs.s3a.access-key
   client.fs.s3a.secret-key
   client.fs.s3a.session.token
   client.fs.s3a.assumed.role.arn
   client.fs.s3a.assumed.role.*
   client.fs.s3a.aws.credentials.provider
   ```
   
   These options are stripped to `fs.s3a.*` and passed to the S3 filesystem plugin. If `client.fs.s3a.access.key` and `client.fs.s3a.secret.key` are configured, `S3FileSystemPlugin` sees static credentials and skips configuring `DynamicTemporaryAWSCredentialsProvider`.
   
   Expected behavior:
   
   - Fluss clients should fail fast when credential-bearing filesystem options are configured through `client.fs.*`.
   - Fluss clients should continue to allow non-credential filesystem runtime options, such as endpoint, region, path-style access, or download tuning options.
   
   Examples of options that should still be allowed:
   
   ```text
   client.fs.s3a.endpoint
   client.fs.s3a.region
   client.fs.s3a.path-style-access
   client.fs.s3a.path.style.access
   client.fs.oss.endpoint
   client.fs.oss.multipart.download.threads
   ```
   
   ### Solution
   
   Add validation before forwarding `client.fs.*` options to `FileSystem.initialize`.
   
   Reject credential-bearing client-side filesystem options fail-fast, at least for S3:
   
   ```text
   client.fs.s3a.aws.credentials.provider
   client.fs.s3a.access.key
   client.fs.s3a.secret.key
   client.fs.s3a.access-key
   client.fs.s3a.secret-key
   client.fs.s3a.session.token
   client.fs.s3a.assumed.role.arn
   client.fs.s3a.assumed.role.*
   ```
   
   The error message should clearly state that filesystem credentials must be configured on Fluss servers, and Fluss clients should use server-issued temporary filesystem tokens.
   
   Suggested tests:
   
   1. Non-credential `client.fs.*` options are still forwarded to the filesystem.
   2. `client.fs.s3a.access.key` and `client.fs.s3a.secret.key` fail fast.
   3. `client.fs.s3a.aws.credentials.provider` fails fast.
   4. `client.fs.s3a.assumed.role.arn` and `client.fs.s3a.assumed.role.*` fail fast.
   5. Normal S3 client access without credential-bearing client options still uses `DynamicTemporaryAWSCredentialsProvider`.
   
   Related issues and PRs:
   
   - #935
   - #938
   
   ### Are you willing to submit a PR?
   
   - [ ] I am willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [client][filesystem] Reject client-side S3 credential options [fluss]

Posted by "litiliu (via GitHub)" <gi...@apache.org>.
litiliu commented on issue #3524:
URL: https://github.com/apache/fluss/issues/3524#issuecomment-4808428364

   Closing this after re-evaluating the scenario.
   
   The issue is mainly a user misconfiguration case: users should not configure S3 credentials such as AK/SK, session token, AssumeRole, or credential provider settings through client.fs.*. If they do, the client-side filesystem can choose a different credential provider path, but that is caused by the invalid configuration itself rather than a normal Fluss client flow.
   
   Given that, adding extra validation here is not clearly necessary


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] [client][filesystem] Reject client-side S3 credential options [fluss]

Posted by "litiliu (via GitHub)" <gi...@apache.org>.
litiliu closed issue #3524: [client][filesystem] Reject client-side S3 credential options
URL: https://github.com/apache/fluss/issues/3524


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@fluss.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org