You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@fluss.apache.org by ab...@apache.org on 2026/07/07 01:36:14 UTC

(fluss) 02/02: [docs] Document S3 custom AWS credentials provider mode

This is an automated email from the ASF dual-hosted git repository.

fresh-borzoni pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fluss.git

commit c5a0c00a22999e375afc2ac3e50f5c6cb26886a1
Author: Anton Borisov <an...@fresha.com>
AuthorDate: Tue Jul 7 02:01:53 2026 +0100

    [docs] Document S3 custom AWS credentials provider mode
---
 .../maintenance/tiered-storage/filesystems/s3.md     | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/website/docs/maintenance/tiered-storage/filesystems/s3.md b/website/docs/maintenance/tiered-storage/filesystems/s3.md
index ca22dd339..311573a59 100644
--- a/website/docs/maintenance/tiered-storage/filesystems/s3.md
+++ b/website/docs/maintenance/tiered-storage/filesystems/s3.md
@@ -86,7 +86,7 @@ Without `s3.assumed.role.arn`, Fluss falls back to `GetSessionToken` (the defaul
 When running Fluss on Kubernetes with [IAM Roles for Service Accounts (IRSA)](https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html) or on EC2 with instance profiles, you can omit `s3.access-key` and `s3.secret-key`. The server will authenticate using the default AWS credential chain.
 
 :::note
-The explicit server-side credential provider mode (`s3.aws.credentials.provider`) only supports providers that return long-term access key credentials and cannot be combined with `s3.assumed.role.arn`. IRSA and instance profile providers return session credentials, so they must use the default AWS credential chain described in this section instead of the explicit provider mode.
+IRSA and instance profiles return session credentials, so they must use this default chain — not the explicit [Custom AWS Credentials Provider](#custom-aws-credentials-provider) mode, which requires long-term credentials.
 :::
 
 In this mode, `s3.assumed.role.arn` is required — the server uses `AssumeRole` to generate temporary credentials for clients (Flink/Spark connectors) that read tiered data from S3.
@@ -104,3 +104,21 @@ The server's IAM role (e.g., the IRSA service account role) must have:
 :::note
 The server authenticates using its own credentials (static keys, IRSA, instance profile, or environment variables) for S3 data access. For delegation, the server calls AssumeRole with the configured `s3.assumed.role.arn`, so clients receive credentials for that role — which can have different permissions (e.g., read-only). Note that the server uses the same identity for both its own S3 access and the STS AssumeRole call. Further decoupling is planned.
 :::
+
+### Custom AWS Credentials Provider
+
+For deployments that manage long-term credentials through a standard AWS SDK or Hadoop S3A credentials provider (for example, a rotating profile), set `s3.aws.credentials.provider` to the provider class. Fluss then treats it as the authoritative server-side credential source, so rotated credentials are picked up without restarting the servers.
+
+```yaml
+remote.data.dir: s3://<your-bucket>/path/to/remote/storage
+s3.region: <your-s3-region>
+s3.aws.credentials.provider: com.amazonaws.auth.profile.ProfileCredentialsProvider
+```
+
+| Configuration | Description |
+|---|---|
+| `s3.aws.credentials.provider` | Fully qualified class name of an AWS SDK / Hadoop S3A credentials provider. It is used both for the server's own S3 access and for resolving the credentials used to mint temporary client credentials via `GetSessionToken`. Takes precedence over `s3.access-key`/`s3.secret-key` when both are configured. |
+
+:::note
+This mode only supports providers that return **long-term** access-key credentials, and it cannot be combined with `s3.assumed.role.arn`. Providers that return session credentials — including IRSA and EC2 instance profiles — must use the [default AWS credential chain](#default-aws-credential-chain-irsa-instance-profiles) instead.
+:::