You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by li...@apache.org on 2026/07/21 14:20:08 UTC
(seatunnel) branch dev updated: [Feature][Connector-V2] Support arbitrary S3A credentials provider for S3File (#11436)
This is an automated email from the ASF dual-hosted git repository.
davidzollo pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new d34446997f [Feature][Connector-V2] Support arbitrary S3A credentials provider for S3File (#11436)
d34446997f is described below
commit d34446997f163557717a9d58b4f43b7def5154f5
Author: FatMan <15...@users.noreply.github.com>
AuthorDate: Tue Jul 21 22:20:01 2026 +0800
[Feature][Connector-V2] Support arbitrary S3A credentials provider for S3File (#11436)
Co-authored-by: chen <77...@user.noreply.gitee.com>
---
docs/en/connectors/sink/S3File.md | 4 +-
docs/en/connectors/source/S3File.md | 2 +-
docs/zh/connectors/sink/S3File.md | 4 +-
docs/zh/connectors/source/S3File.md | 2 +-
.../file/s3/config/S3FileBaseOptions.java | 57 +++++-
.../seatunnel/file/s3/config/S3HadoopConf.java | 142 ++++++++++++-
.../seatunnel/file/s3/sink/S3FileSinkFactory.java | 6 +-
.../file/s3/source/S3FileSourceFactory.java | 6 +-
.../seatunnel/file/s3/S3FileFactoryTest.java | 32 +++
.../seatunnel/file/s3/config/S3HadoopConfTest.java | 224 +++++++++++++++++++++
.../seatunnel/hive/storage/S3Storage.java | 5 +-
.../seatunnel/hive/storage/S3StorageTest.java | 8 +-
.../redshift/sink/S3RedshiftSinkFactory.java | 6 +-
13 files changed, 473 insertions(+), 25 deletions(-)
diff --git a/docs/en/connectors/sink/S3File.md b/docs/en/connectors/sink/S3File.md
index a72a8fab59..a254445da4 100644
--- a/docs/en/connectors/sink/S3File.md
+++ b/docs/en/connectors/sink/S3File.md
@@ -110,7 +110,7 @@ If write to `csv`, `text` file type, All column will be string.
| tmp_path | string | no | /tmp/seatunnel | The result file will write to a tmp path first and then use `mv` to submit tmp dir to target dir. Need a S3 dir. |
| bucket | string | yes | - | |
| fs.s3a.endpoint | string | yes | - | |
-| fs.s3a.aws.credentials.provider | string | yes | com.amazonaws.auth.InstanceProfileCredentialsProvider | The way to authenticate s3a. We only support `org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider` and `com.amazonaws.auth.InstanceProfileCredentialsProvider` now. |
+| fs.s3a.aws.credentials.provider | string | yes | com.amazonaws.auth.InstanceProfileCredentialsProvider | The fully-qualified class name of the S3A credentials provider passed through to Hadoop. Besides the two well-known values `org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider` (static `access_key`/`secret_key`) and `com.amazonaws.auth.InstanceProfileCredentialsProvider` (default), any S3A credentials provider class available on the classpath is accepted, for example c [...]
| access_key | string | no | - | Only used when fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider |
| secret_key | string | no | - | Only used when fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider |
| custom_filename | boolean | no | false | Whether you need custom the filename |
@@ -561,4 +561,4 @@ LocalFile {
## Changelog
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/en/connectors/source/S3File.md b/docs/en/connectors/source/S3File.md
index 6b510045e1..e2f0d1ac43 100644
--- a/docs/en/connectors/source/S3File.md
+++ b/docs/en/connectors/source/S3File.md
@@ -196,7 +196,7 @@ If you assign file type to `parquet` `orc`, schema option not required, connecto
| file_format_type | string | yes | - | File type, supported as the following file types: `text` `csv` `parquet` `orc` `json` `excel` `xml` `binary` `markdown` `pdf` [...]
| bucket | string | yes | - | The bucket address of s3 file system, for example: `s3n://seatunnel-test`, if you use `s3a` protocol, this parameter should be `s3a://seatunnel-test`. [...]
| fs.s3a.endpoint | string | yes | - | fs s3a endpoint [...]
-| fs.s3a.aws.credentials.provider | string | yes | com.amazonaws.auth.InstanceProfileCredentialsProvider | The way to authenticate s3a. We only support `org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider` and `com.amazonaws.auth.InstanceProfileCredentialsProvider` now. More information about the credential provider you can see [Hadoop AWS Document](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#Simple_name.2Fsecret_credentials_with_SimpleAWSCredenti [...]
+| fs.s3a.aws.credentials.provider | string | yes | com.amazonaws.auth.InstanceProfileCredentialsProvider | The fully-qualified class name of the S3A credentials provider passed through to Hadoop. Besides the two well-known values `org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider` (static `access_key`/`secret_key`) and `com.amazonaws.auth.InstanceProfileCredentialsProvider` (default), any S3A credentials provider class available on the classpath is accepted, for example contain [...]
| read_columns | list | no | - | The read column list of the data source, user can use it to implement field projection. The file type supported column projection as the following shown: `text` `csv` `parquet` `orc` `json` `excel` `xml` . If the user wants to use this feature when reading `text` `json` `csv` files, the "schema" option must be configured. [...]
| access_key | string | no | - | Only used when `fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider ` [...]
| secret_key | string | no | - | Only used when `fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider ` [...]
diff --git a/docs/zh/connectors/sink/S3File.md b/docs/zh/connectors/sink/S3File.md
index 17808ae980..b06abaa3a2 100644
--- a/docs/zh/connectors/sink/S3File.md
+++ b/docs/zh/connectors/sink/S3File.md
@@ -109,7 +109,7 @@ import ChangeLog from '../changelog/connector-file-s3.md';
| tmp_path | string | 否 | /tmp/seatunnel | 结果文件将首先写入临时路径,然后使用 `mv` 将临时目录提交到目标目录。需要一个 S3 目录。 |
| bucket | string | 是 | - | |
| fs.s3a.endpoint | string | 是 | - | |
-| fs.s3a.aws.credentials.provider | string | 是 | com.amazonaws.auth.InstanceProfileCredentialsProvider | 认证 s3a 的方式。目前仅支持 `org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider` 和 `com.amazonaws.auth.InstanceProfileCredentialsProvider`。 |
+| fs.s3a.aws.credentials.provider | string | 是 | com.amazonaws.auth.InstanceProfileCredentialsProvider | 透传给 Hadoop 的 S3A 凭据提供程序的全限定类名。除了两个常用值 `org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider`(使用静态 `access_key`/`secret_key`)和 `com.amazonaws.auth.InstanceProfileCredentialsProvider`(默认值)之外,任何 classpath 上可用的 S3A 凭据提供程序类都可以使用,例如基于容器的 `com.amazonaws.auth.ContainerCredentialsProvider` 或自定义提供程序。该类必须实现 `com.amazonaws.auth.AWSCredentialsProvider` 接口,并提供 Hadoop 3.1.4 支持的任一创建方式:公共 ` [...]
| access_key | string | 否 | - | 仅当 fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider 时使用 |
| secret_key | string | 否 | - | 仅当 fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider 时使用 |
| custom_filename | boolean | 否 | false | 是否需要自定义文件名 |
@@ -549,4 +549,4 @@ LocalFile {
## 变更日志
-<ChangeLog />
\ No newline at end of file
+<ChangeLog />
diff --git a/docs/zh/connectors/source/S3File.md b/docs/zh/connectors/source/S3File.md
index 9e5768fefa..3367b8eac7 100644
--- a/docs/zh/connectors/source/S3File.md
+++ b/docs/zh/connectors/source/S3File.md
@@ -196,7 +196,7 @@ schema {
| file_format_type | string | 是 | - | 文件类型,支持以下文件类型:`text` `csv` `parquet` `orc` `json` `excel` `xml` `binary` `markdown` `pdf` |
| bucket | string | 是 | - | s3文件系统的bucket地址,例如:`s3n://seatunnel-test`,如果您使用`s3a`协议,此参数应为`s3a://seatunnel-test`。 |
| fs.s3a.endpoint | string | 是 | - | fs s3a端点 |
-| fs.s3a.aws.credentials.provider | string | 是 | com.amazonaws.auth.InstanceProfileCredentialsProvider | s3a的认证方式。我们目前只支持`org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider`和`com.amazonaws.auth.InstanceProfileCredentialsProvider`。有关凭据提供程序的更多信息,您可以查看[Hadoop AWS文档](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#Simple_name.2Fsecret_credentials_with_SimpleAWSCredentialsProvider.2A) |
+| fs.s3a.aws.credentials.provider | string | 是 | com.amazonaws.auth.InstanceProfileCredentialsProvider | 透传给 Hadoop 的 S3A 凭据提供程序的全限定类名。除了两个常用值 `org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider`(使用静态 `access_key`/`secret_key`)和 `com.amazonaws.auth.InstanceProfileCredentialsProvider`(默认值)之外,任何 classpath 上可用的 S3A 凭据提供程序类都可以使用,例如基于容器的 `com.amazonaws.auth.ContainerCredentialsProvider` 或自定义提供程序。该类必须实现 `com.amazonaws.auth.AWSCredentialsProvider` 接口,并提供 Hadoop 3.1.4 支持的任一创建方式:公共 `(java. [...]
| read_columns | list | 否 | - | 数据源的读取列列表,用户可以使用它来实现字段投影。支持列投影的文件类型如下所示:`text` `csv` `parquet` `orc` `json` `excel` `xml`。如果用户想在读取`text` `json` `csv`文件时使用此功能,必须配置"schema"选项。 |
| access_key | string | 否 | - | 仅在`fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider`时使用 |
| secret_key | string | 否 | - | 仅在`fs.s3a.aws.credentials.provider = org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider`时使用 |
diff --git a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3FileBaseOptions.java b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3FileBaseOptions.java
index 9d0a45d0cc..02b5e787d9 100644
--- a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3FileBaseOptions.java
+++ b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3FileBaseOptions.java
@@ -42,12 +42,54 @@ public class S3FileBaseOptions extends FileBaseSourceOptions {
.noDefaultValue()
.withDescription("fs s3a endpoint");
+ /**
+ * The class name of the {@code SimpleAWSCredentialsProvider}, which authenticates with a static
+ * {@code access_key} / {@code secret_key} pair.
+ */
+ public static final String SIMPLE_AWS_CREDENTIALS_PROVIDER =
+ "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider";
+
+ /**
+ * The class name of the {@code InstanceProfileCredentialsProvider}, which resolves credentials
+ * from the runtime environment (for example an EC2 instance profile).
+ */
+ public static final String INSTANCE_PROFILE_CREDENTIALS_PROVIDER =
+ "com.amazonaws.auth.InstanceProfileCredentialsProvider";
+
+ /**
+ * The legacy typed S3A credentials provider option.
+ *
+ * @deprecated Use {@link #S3A_AWS_CREDENTIALS_PROVIDER_CLASS}. This field retains its original
+ * generic type so downstream code compiled against earlier releases remains source and
+ * binary compatible.
+ */
+ @Deprecated
public static final Option<S3aAwsCredentialsProvider> S3A_AWS_CREDENTIALS_PROVIDER =
Options.key("fs.s3a.aws.credentials.provider")
.enumType(S3aAwsCredentialsProvider.class)
.defaultValue(S3aAwsCredentialsProvider.InstanceProfileCredentialsProvider)
.withDescription("s3a aws credentials provider");
+ /**
+ * The S3A credentials provider class name passed through to Hadoop. Any fully-qualified S3A
+ * credentials provider class available on the classpath is accepted, so container-oriented
+ * providers (for example {@code com.amazonaws.auth.ContainerCredentialsProvider}) and custom
+ * providers can be used in addition to the two well-known values {@link
+ * #SIMPLE_AWS_CREDENTIALS_PROVIDER} and {@link #INSTANCE_PROFILE_CREDENTIALS_PROVIDER}. Hadoop
+ * also accepts a comma- or newline-separated chain of provider classes. The class must be
+ * present on the runtime classpath of every node running the S3A filesystem (for example under
+ * {@code ${SEATUNNEL_HOME}/lib}).
+ */
+ public static final Option<String> S3A_AWS_CREDENTIALS_PROVIDER_CLASS =
+ Options.key(S3A_AWS_CREDENTIALS_PROVIDER.key())
+ .stringType()
+ .defaultValue(INSTANCE_PROFILE_CREDENTIALS_PROVIDER)
+ .withDescription(
+ "The fully-qualified class name of the S3A credentials provider. "
+ + "Defaults to "
+ + INSTANCE_PROFILE_CREDENTIALS_PROVIDER
+ + ". The class must be present on the classpath.");
+
/**
* The current key for that config option. if you need to add a new option, you can add it here
* and refer to this:
@@ -62,12 +104,21 @@ public class S3FileBaseOptions extends FileBaseSourceOptions {
.noDefaultValue()
.withDescription("S3 properties");
+ /**
+ * The set of S3A credentials providers accepted by the legacy {@link
+ * #S3A_AWS_CREDENTIALS_PROVIDER} option.
+ *
+ * @deprecated Use {@link #S3A_AWS_CREDENTIALS_PROVIDER_CLASS} with the {@link
+ * #SIMPLE_AWS_CREDENTIALS_PROVIDER} and {@link #INSTANCE_PROFILE_CREDENTIALS_PROVIDER}
+ * class-name constants. This enum remains the type of the deprecated compatibility option.
+ */
+ @Deprecated
public enum S3aAwsCredentialsProvider {
- SimpleAWSCredentialsProvider("org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider"),
+ SimpleAWSCredentialsProvider(SIMPLE_AWS_CREDENTIALS_PROVIDER),
- InstanceProfileCredentialsProvider("com.amazonaws.auth.InstanceProfileCredentialsProvider");
+ InstanceProfileCredentialsProvider(INSTANCE_PROFILE_CREDENTIALS_PROVIDER);
- private String provider;
+ private final String provider;
S3aAwsCredentialsProvider(String provider) {
this.provider = provider;
diff --git a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3HadoopConf.java b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3HadoopConf.java
index a9307f5320..99f4a8cbe0 100644
--- a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3HadoopConf.java
+++ b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3HadoopConf.java
@@ -20,10 +20,17 @@ package org.apache.seatunnel.connectors.seatunnel.file.s3.config;
import org.apache.seatunnel.api.configuration.ReadonlyConfig;
import org.apache.seatunnel.connectors.seatunnel.file.config.HadoopConf;
+import org.apache.hadoop.util.StringUtils;
+
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.lang.reflect.Modifier;
import java.util.HashMap;
import java.util.Map;
public class S3HadoopConf extends HadoopConf {
+ private static final Logger LOGGER = LoggerFactory.getLogger(S3HadoopConf.class);
private static final String HDFS_S3N_IMPL = "org.apache.hadoop.fs.s3native.NativeS3FileSystem";
private static final String HDFS_S3A_IMPL = "org.apache.hadoop.fs.s3a.S3AFileSystem";
protected static final String S3A_SCHEMA = "s3a";
@@ -62,9 +69,11 @@ public class S3HadoopConf extends HadoopConf {
.forEach((key, value) -> s3Options.put(key, String.valueOf(value)));
}
+ String credentialsProvider =
+ config.get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS);
+ checkCredentialsProviders(credentialsProvider);
s3Options.put(
- S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER.key(),
- config.get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER).getProvider());
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), credentialsProvider);
s3Options.put(
S3FileBaseOptions.FS_S3A_ENDPOINT.key(),
config.get(S3FileBaseOptions.FS_S3A_ENDPOINT));
@@ -97,4 +106,133 @@ public class S3HadoopConf extends HadoopConf {
s3Options.put("fs.s3n.awsAccessKeyId", accessKey);
s3Options.put("fs.s3n.awsSecretAccessKey", secretKey);
}
+
+ /**
+ * The AWS credentials provider contract every configured provider class must implement. Hadoop
+ * S3A's {@code S3AUtils.createAWSCredentialProvider} performs the exact same {@code
+ * AWSCredentialsProvider.isAssignableFrom(clazz)} check (and rejects abstract classes) before
+ * it reflectively instantiates the class, so validating it here only moves that failure
+ * earlier, from an opaque worker-side {@code IOException} to an actionable config-parse-time
+ * error. It never rejects anything Hadoop itself would have accepted.
+ */
+ private static final String AWS_CREDENTIALS_PROVIDER_INTERFACE =
+ "com.amazonaws.auth.AWSCredentialsProvider";
+
+ /**
+ * Best-effort eager validation of the configured S3A credentials provider value on the node
+ * that builds the Hadoop configuration. Hadoop accepts a comma- or newline-separated chain of
+ * provider classes, so each entry is validated independently.
+ *
+ * <p>Provider names are parsed with Hadoop's own {@link StringUtils#getTrimmedStrings(String)}
+ * utility. This preserves Hadoop 3.1.4 behavior for whitespace, newlines, and trailing
+ * separators while still allowing an interior empty entry such as {@code "A,,B"} to fail with
+ * an actionable error.
+ *
+ * <p>The provider is ultimately instantiated on every worker by Hadoop S3A, and the classpath
+ * of the node building this configuration (client/coordinator, or a module embedding this
+ * connector) may legitimately differ from the workers' — the provider jar only has to be
+ * present where S3A actually runs (for example under {@code ${SEATUNNEL_HOME}/lib} on every
+ * cluster node). A class that cannot be resolved here is therefore only logged as a warning,
+ * never rejected. Only a class that resolves but does not implement the AWS credentials
+ * provider contract, or a malformed (empty) chain entry, fails fast because that configuration
+ * cannot work on any node.
+ *
+ * @param credentialsProvider the raw option value (single class or comma-separated chain)
+ */
+ private static void checkCredentialsProviders(String credentialsProvider) {
+ String[] providerClassNames = StringUtils.getTrimmedStrings(credentialsProvider);
+ for (String providerClassName : providerClassNames) {
+ if (providerClassName.isEmpty()) {
+ throw new IllegalArgumentException(
+ String.format(
+ "The S3A credentials provider chain configured via '%s' contains an "
+ + "empty class name ('%s'). Please ensure every entry between "
+ + "separators is a fully-qualified provider class name.",
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(),
+ credentialsProvider));
+ }
+ checkCredentialsProviderClass(providerClassName);
+ }
+ }
+
+ /**
+ * Validates a single S3A credentials provider class as far as the local classpath allows.
+ *
+ * <p>The lookup tries the thread context classloader first, then falls back to the classloader
+ * that loaded this connector. Under the Zeta plugin classloader the context classloader may be
+ * {@code null} or unrelated to the connector, so the fallback ensures a provider bundled with
+ * the connector (including the default {@link
+ * S3FileBaseOptions#INSTANCE_PROFILE_CREDENTIALS_PROVIDER}) is not rejected by mistake.
+ *
+ * @param providerClassName the fully-qualified credentials provider class name
+ */
+ private static void checkCredentialsProviderClass(String providerClassName) {
+ for (ClassLoader classLoader : credentialsProviderClassLoaders()) {
+ if (classLoader == null) {
+ continue;
+ }
+ Class<?> providerClass;
+ Class<?> providerInterface;
+ try {
+ providerClass = Class.forName(providerClassName, false, classLoader);
+ providerInterface =
+ Class.forName(AWS_CREDENTIALS_PROVIDER_INTERFACE, false, classLoader);
+ } catch (ClassNotFoundException ignored) {
+ // Both classes must resolve through the same loader before assignability can be
+ // checked. Try the next loader instead of treating a skipped check as success.
+ continue;
+ }
+ assertImplementsCredentialsProvider(providerClass, providerInterface);
+ return;
+ }
+ LOGGER.warn(
+ "The S3A credentials provider class '{}' configured via '{}' could not be fully "
+ + "validated on this node because the provider class or '{}' is not visible "
+ + "through the available classloaders. The job will fail at runtime unless "
+ + "the provider jar is available on every node running the S3A filesystem "
+ + "(for example under ${SEATUNNEL_HOME}/lib).",
+ providerClassName,
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(),
+ AWS_CREDENTIALS_PROVIDER_INTERFACE);
+ }
+
+ /**
+ * Asserts the resolved provider class can actually be used by Hadoop S3A: it must implement
+ * {@link #AWS_CREDENTIALS_PROVIDER_INTERFACE} and must not be abstract. These are exactly the
+ * two conditions {@code S3AUtils.createAWSCredentialProvider} enforces before instantiation, so
+ * this check never rejects a class Hadoop would have accepted — it only surfaces the failure at
+ * config-parse time with an actionable message instead of an opaque worker-side error.
+ *
+ * <p>The caller resolves both classes through the same candidate classloader. If either class
+ * is unavailable it tries the next loader, so an isolated thread context classloader cannot
+ * bypass validation when the connector classloader can perform it.
+ */
+ private static void assertImplementsCredentialsProvider(
+ Class<?> providerClass, Class<?> providerInterface) {
+ if (!providerInterface.isAssignableFrom(providerClass)) {
+ throw new IllegalArgumentException(
+ String.format(
+ "The S3A credentials provider class '%s' does not implement '%s'. "
+ + "Please check the value of '%s' and configure a class that "
+ + "implements the AWS credentials provider contract.",
+ providerClass.getName(),
+ AWS_CREDENTIALS_PROVIDER_INTERFACE,
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key()));
+ }
+ if (Modifier.isAbstract(providerClass.getModifiers())) {
+ throw new IllegalArgumentException(
+ String.format(
+ "The S3A credentials provider class '%s' configured via '%s' is "
+ + "abstract and cannot be instantiated. Please configure a "
+ + "concrete AWS credentials provider class.",
+ providerClass.getName(),
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key()));
+ }
+ }
+
+ private static ClassLoader[] credentialsProviderClassLoaders() {
+ return new ClassLoader[] {
+ Thread.currentThread().getContextClassLoader(), S3HadoopConf.class.getClassLoader()
+ };
+ }
}
diff --git a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/sink/S3FileSinkFactory.java b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/sink/S3FileSinkFactory.java
index 28c6390186..64b6100974 100644
--- a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/sink/S3FileSinkFactory.java
+++ b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/sink/S3FileSinkFactory.java
@@ -47,12 +47,12 @@ public class S3FileSinkFactory implements TableSinkFactory {
.required(S3FileSinkOptions.FILE_PATH)
.required(S3FileSinkOptions.S3_BUCKET)
.required(S3FileSinkOptions.FS_S3A_ENDPOINT)
- .required(S3FileSinkOptions.S3A_AWS_CREDENTIALS_PROVIDER)
+ .required(S3FileSinkOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS)
.optional(FileBaseSinkOptions.SCHEMA_SAVE_MODE)
.optional(FileBaseSinkOptions.DATA_SAVE_MODE)
.conditional(
- S3FileSinkOptions.S3A_AWS_CREDENTIALS_PROVIDER,
- S3FileSinkOptions.S3aAwsCredentialsProvider.SimpleAWSCredentialsProvider,
+ S3FileSinkOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS,
+ S3FileSinkOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER,
S3FileSinkOptions.S3_ACCESS_KEY,
S3FileSinkOptions.S3_SECRET_KEY)
.optional(S3FileSinkOptions.S3_PROPERTIES)
diff --git a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/source/S3FileSourceFactory.java b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/source/S3FileSourceFactory.java
index 89194eff0c..7a8bc1a679 100644
--- a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/source/S3FileSourceFactory.java
+++ b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/s3/source/S3FileSourceFactory.java
@@ -57,10 +57,10 @@ public class S3FileSourceFactory implements TableSourceFactory {
.required(S3FileSourceOptions.FILE_FORMAT_TYPE)
.required(S3FileSourceOptions.S3_BUCKET)
.required(S3FileSourceOptions.FS_S3A_ENDPOINT)
- .required(S3FileSourceOptions.S3A_AWS_CREDENTIALS_PROVIDER)
+ .required(S3FileSourceOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS)
.conditional(
- S3FileSourceOptions.S3A_AWS_CREDENTIALS_PROVIDER,
- S3FileSourceOptions.S3aAwsCredentialsProvider.SimpleAWSCredentialsProvider,
+ S3FileSourceOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS,
+ S3FileSourceOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER,
S3FileSourceOptions.S3_ACCESS_KEY,
S3FileSourceOptions.S3_SECRET_KEY)
.optional(S3FileSourceOptions.S3_PROPERTIES)
diff --git a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/s3/S3FileFactoryTest.java b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/s3/S3FileFactoryTest.java
index 393f8dc7c3..756fbe7e8b 100644
--- a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/s3/S3FileFactoryTest.java
+++ b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/s3/S3FileFactoryTest.java
@@ -23,6 +23,7 @@ import org.apache.seatunnel.api.configuration.util.Expression;
import org.apache.seatunnel.api.configuration.util.OptionRule;
import org.apache.seatunnel.api.configuration.util.RequiredOption;
import org.apache.seatunnel.connectors.seatunnel.file.config.FileBaseSourceOptions;
+import org.apache.seatunnel.connectors.seatunnel.file.s3.config.S3FileBaseOptions;
import org.apache.seatunnel.connectors.seatunnel.file.s3.sink.S3FileSinkFactory;
import org.apache.seatunnel.connectors.seatunnel.file.s3.source.S3FileSourceFactory;
@@ -62,6 +63,37 @@ class S3FileFactoryTest {
"S3File source optionRule should expose file_split_size when enable_file_split=true");
}
+ @Test
+ void credentialsProviderConditionalRequiresAccessKeys() {
+ OptionRule sourceRule = new S3FileSourceFactory().optionRule();
+ Assertions.assertTrue(
+ hasConditionalRequiredOption(
+ sourceRule,
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS,
+ S3FileBaseOptions.S3_ACCESS_KEY),
+ "S3File source optionRule should require access_key when the credentials provider is SimpleAWSCredentialsProvider");
+ Assertions.assertTrue(
+ hasConditionalRequiredOption(
+ sourceRule,
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS,
+ S3FileBaseOptions.S3_SECRET_KEY),
+ "S3File source optionRule should require secret_key when the credentials provider is SimpleAWSCredentialsProvider");
+
+ OptionRule sinkRule = new S3FileSinkFactory().optionRule();
+ Assertions.assertTrue(
+ hasConditionalRequiredOption(
+ sinkRule,
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS,
+ S3FileBaseOptions.S3_ACCESS_KEY),
+ "S3File sink optionRule should require access_key when the credentials provider is SimpleAWSCredentialsProvider");
+ Assertions.assertTrue(
+ hasConditionalRequiredOption(
+ sinkRule,
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS,
+ S3FileBaseOptions.S3_SECRET_KEY),
+ "S3File sink optionRule should require secret_key when the credentials provider is SimpleAWSCredentialsProvider");
+ }
+
private static boolean optionRuleContains(OptionRule rule, Option<?> option) {
if (rule.getOptionalOptions().contains(option)) {
return true;
diff --git a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3HadoopConfTest.java b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3HadoopConfTest.java
index f090795d28..2f764d178e 100644
--- a/seatunnel-connectors-v2/connector-file/connector-file-s3/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3HadoopConfTest.java
+++ b/seatunnel-connectors-v2/connector-file/connector-file-s3/src/test/java/org/apache/seatunnel/connectors/seatunnel/file/s3/config/S3HadoopConfTest.java
@@ -45,4 +45,228 @@ public class S3HadoopConfTest {
conf = S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config));
Assertions.assertFalse(conf.getExtraOptions().containsKey("fs.s3n.awsAccessKeyId"));
}
+
+ @Test
+ void testDefaultCredentialsProvider() {
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ HadoopConf conf = S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config));
+ Assertions.assertEquals(
+ S3FileBaseOptions.INSTANCE_PROFILE_CREDENTIALS_PROVIDER,
+ conf.getExtraOptions()
+ .get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key()));
+ }
+
+ @Test
+ void testLegacyCredentialsProviderOptionTypeIsPreserved() {
+ Map<String, Object> config = new HashMap<>();
+ config.put(
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER.key(),
+ S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER);
+
+ S3FileBaseOptions.S3aAwsCredentialsProvider provider =
+ ReadonlyConfig.fromMap(config).get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER);
+
+ Assertions.assertEquals(
+ S3FileBaseOptions.S3aAwsCredentialsProvider.SimpleAWSCredentialsProvider, provider);
+ }
+
+ @Test
+ void testCustomCredentialsProviderIsPassedThrough() {
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(),
+ S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER);
+ config.put("access_key", "access_key");
+ config.put("secret_key", "secret_key");
+ HadoopConf conf = S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config));
+ Assertions.assertEquals(
+ S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER,
+ conf.getExtraOptions()
+ .get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key()));
+ }
+
+ @Test
+ void testUnknownCredentialsProviderClassIsPassedThroughWithWarning() {
+ // A class that is not resolvable on this node may still exist on the worker classpath
+ // where Hadoop S3A actually instantiates it, so it must pass through instead of failing.
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(),
+ "com.example.NonExistentCredentialsProvider");
+ HadoopConf conf = S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config));
+ Assertions.assertEquals(
+ "com.example.NonExistentCredentialsProvider",
+ conf.getExtraOptions()
+ .get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key()));
+ }
+
+ @Test
+ void testNonCredentialsProviderClassFails() {
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), "java.lang.String");
+ IllegalArgumentException exception =
+ Assertions.assertThrows(
+ IllegalArgumentException.class,
+ () -> S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config)));
+ Assertions.assertTrue(exception.getMessage().contains("does not implement"));
+ }
+
+ @Test
+ void testNonProviderCannotBypassValidationWithIsolatedContextClassLoader() {
+ ClassLoader original = Thread.currentThread().getContextClassLoader();
+ try {
+ Thread.currentThread().setContextClassLoader(new ClassLoader(null) {});
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), "java.lang.String");
+
+ Assertions.assertThrows(
+ IllegalArgumentException.class,
+ () -> S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config)));
+ } finally {
+ Thread.currentThread().setContextClassLoader(original);
+ }
+ }
+
+ @Test
+ void testProviderChainIsPassedThrough() {
+ String chain =
+ S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER
+ + ","
+ + S3FileBaseOptions.INSTANCE_PROFILE_CREDENTIALS_PROVIDER;
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), chain);
+ config.put("access_key", "access_key");
+ config.put("secret_key", "secret_key");
+ HadoopConf conf = S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config));
+ Assertions.assertEquals(
+ chain,
+ conf.getExtraOptions()
+ .get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key()));
+ }
+
+ @Test
+ void testProviderChainWithNonProviderClassFails() {
+ // Chain entries are validated independently; a resolvable class that is not a credentials
+ // provider still fails fast because it cannot work on any node.
+ String chain = S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER + ",java.lang.String";
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), chain);
+ config.put("access_key", "access_key");
+ config.put("secret_key", "secret_key");
+ IllegalArgumentException exception =
+ Assertions.assertThrows(
+ IllegalArgumentException.class,
+ () -> S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config)));
+ Assertions.assertTrue(exception.getMessage().contains("does not implement"));
+ }
+
+ @Test
+ void testProviderChainWithWhitespaceIsPassedThrough() {
+ // Hadoop trims whitespace around commas, so "A, B" is a valid chain; the raw value
+ // (including the space) is passed through unchanged for Hadoop to parse.
+ String chain =
+ S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER
+ + ", "
+ + S3FileBaseOptions.INSTANCE_PROFILE_CREDENTIALS_PROVIDER;
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), chain);
+ config.put("access_key", "access_key");
+ config.put("secret_key", "secret_key");
+ HadoopConf conf = S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config));
+ Assertions.assertEquals(
+ chain,
+ conf.getExtraOptions()
+ .get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key()));
+ }
+
+ @Test
+ void testProviderChainWithNewlineValidatesEachEntry() {
+ String chain = S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER + "\njava.lang.String";
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), chain);
+ config.put("access_key", "access_key");
+ config.put("secret_key", "secret_key");
+
+ IllegalArgumentException exception =
+ Assertions.assertThrows(
+ IllegalArgumentException.class,
+ () -> S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config)));
+ Assertions.assertTrue(exception.getMessage().contains("does not implement"));
+ }
+
+ @Test
+ void testProviderChainWithEmptySegmentFails() {
+ String chain =
+ S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER
+ + ",,"
+ + S3FileBaseOptions.INSTANCE_PROFILE_CREDENTIALS_PROVIDER;
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), chain);
+ config.put("access_key", "access_key");
+ config.put("secret_key", "secret_key");
+ IllegalArgumentException exception =
+ Assertions.assertThrows(
+ IllegalArgumentException.class,
+ () -> S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config)));
+ Assertions.assertTrue(exception.getMessage().contains("empty class name"));
+ }
+
+ @Test
+ void testProviderChainWithTrailingCommaMatchesHadoopSemantics() {
+ String chain = S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER + ",";
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), chain);
+ config.put("access_key", "access_key");
+ config.put("secret_key", "secret_key");
+
+ org.apache.hadoop.conf.Configuration hadoopConfig =
+ new org.apache.hadoop.conf.Configuration(false);
+ hadoopConfig.set(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(), chain);
+ Assertions.assertEquals(
+ 1,
+ hadoopConfig.getClasses(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key())
+ .length);
+
+ HadoopConf conf = S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config));
+ Assertions.assertEquals(
+ chain,
+ conf.getExtraOptions()
+ .get(S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key()));
+ }
+
+ @Test
+ void testAbstractCredentialsProviderClassFails() {
+ // An abstract class implementing the provider contract passes the isAssignableFrom check
+ // but cannot be instantiated by Hadoop S3A, which rejects it before construction; the
+ // eager check mirrors that and fails fast with an actionable message.
+ Map<String, Object> config = new HashMap<>();
+ config.put("bucket", "test");
+ config.put(
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(),
+ AbstractTestCredentialsProvider.class.getName());
+ IllegalArgumentException exception =
+ Assertions.assertThrows(
+ IllegalArgumentException.class,
+ () -> S3HadoopConf.buildWithReadOnlyConfig(ReadonlyConfig.fromMap(config)));
+ Assertions.assertTrue(exception.getMessage().contains("abstract"));
+ }
+
+ /**
+ * An abstract class that implements the AWS credentials provider contract. Used to verify the
+ * eager validation rejects abstract classes, matching Hadoop {@code S3AUtils} behavior.
+ */
+ abstract static class AbstractTestCredentialsProvider
+ implements com.amazonaws.auth.AWSCredentialsProvider {}
}
diff --git a/seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/S3Storage.java b/seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/S3Storage.java
index fbb3642307..90d9e9deed 100644
--- a/seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/S3Storage.java
+++ b/seatunnel-connectors-v2/connector-hive/src/main/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/S3Storage.java
@@ -38,10 +38,11 @@ public class S3Storage extends AbstractStorage {
Config config = fillBucket(readonlyConfig, configuration);
config =
config.withValue(
- S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER.key(),
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS.key(),
ConfigValueFactory.fromAnyRef(
configuration.get(
- S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER.key())));
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS
+ .key())));
config =
config.withValue(
S3FileBaseOptions.FS_S3A_ENDPOINT.key(),
diff --git a/seatunnel-connectors-v2/connector-hive/src/test/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/S3StorageTest.java b/seatunnel-connectors-v2/connector-hive/src/test/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/S3StorageTest.java
index efad7a45a7..8afb3a73bc 100644
--- a/seatunnel-connectors-v2/connector-hive/src/test/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/S3StorageTest.java
+++ b/seatunnel-connectors-v2/connector-hive/src/test/java/org/apache/seatunnel/connectors/seatunnel/hive/storage/S3StorageTest.java
@@ -33,7 +33,7 @@ import java.util.HashMap;
public class S3StorageTest {
private static final String TEST_S3_CREDENTIALS_PROVIDER =
- S3FileBaseOptions.S3aAwsCredentialsProvider.SimpleAWSCredentialsProvider.getProvider();
+ S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER;
private static final ReadonlyConfig S3A =
ReadonlyConfig.fromMap(
@@ -47,7 +47,8 @@ public class S3StorageTest {
S3FileBaseOptions.S3_BUCKET.key(),
"s3a://my_bucket");
put(
- S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER
+ S3FileBaseOptions
+ .S3A_AWS_CREDENTIALS_PROVIDER_CLASS
.key(),
TEST_S3_CREDENTIALS_PROVIDER);
put(
@@ -70,7 +71,8 @@ public class S3StorageTest {
S3FileBaseOptions.S3_BUCKET.key(),
"s3://my_bucket");
put(
- S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER
+ S3FileBaseOptions
+ .S3A_AWS_CREDENTIALS_PROVIDER_CLASS
.key(),
TEST_S3_CREDENTIALS_PROVIDER);
put(S3FileBaseOptions.FS_S3A_ENDPOINT.key(), "test");
diff --git a/seatunnel-connectors-v2/connector-s3-redshift/src/main/java/org/apache/seatunnel/connectors/seatunnel/redshift/sink/S3RedshiftSinkFactory.java b/seatunnel-connectors-v2/connector-s3-redshift/src/main/java/org/apache/seatunnel/connectors/seatunnel/redshift/sink/S3RedshiftSinkFactory.java
index bbd7f66cda..5fc99e9180 100644
--- a/seatunnel-connectors-v2/connector-s3-redshift/src/main/java/org/apache/seatunnel/connectors/seatunnel/redshift/sink/S3RedshiftSinkFactory.java
+++ b/seatunnel-connectors-v2/connector-s3-redshift/src/main/java/org/apache/seatunnel/connectors/seatunnel/redshift/sink/S3RedshiftSinkFactory.java
@@ -48,10 +48,10 @@ public class S3RedshiftSinkFactory implements TableSinkFactory {
S3RedshiftSinkOptions.JDBC_PASSWORD,
S3RedshiftSinkOptions.EXECUTE_SQL,
FileBaseSourceOptions.FILE_PATH,
- S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER)
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS)
.conditional(
- S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER,
- S3FileBaseOptions.S3aAwsCredentialsProvider.SimpleAWSCredentialsProvider,
+ S3FileBaseOptions.S3A_AWS_CREDENTIALS_PROVIDER_CLASS,
+ S3FileBaseOptions.SIMPLE_AWS_CREDENTIALS_PROVIDER,
S3FileBaseOptions.S3_ACCESS_KEY,
S3FileBaseOptions.S3_SECRET_KEY)
.optional(S3FileBaseOptions.S3_PROPERTIES)