You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Maomao Min (Jira)" <ji...@apache.org> on 2023/09/26 08:01:00 UTC

[jira] [Created] (FLINK-33157) Support AWS SDK V2 for Flink's S3 FileSystem

Maomao Min created FLINK-33157:
----------------------------------

             Summary: Support AWS SDK V2 for Flink's S3 FileSystem
                 Key: FLINK-33157
                 URL: https://issues.apache.org/jira/browse/FLINK-33157
             Project: Flink
          Issue Type: Improvement
          Components: FileSystems
            Reporter: Maomao Min


Currently, Flink's S3 FileSystem only supports AWS SDK V1. It's recommended by AWS to use AWS SDK V2 for improvements and longer maintenance support. To complete this work, the following modules need to be updated:
 * flink-s3-fs-base
 * flink-s3-fs-hadoop
 * flink-s3-fs-presto

The Hadoop has already added support for AWS SDK V2, but this is not the case for Presto. To unblock component flink-s3-fs-presto for AWS SDK V2 support, we might have the following options:
 # Flink can have a copy of Presto's S3 FileSystem with AWS SDK V2 support added.
 # Add AWS SDK V2 support in Presto's S3 FileSystem in Presto and use new Presto version with this feature in Flink.
 # Use Trino's S3FileSystem, which has already supported AWS SDK V2, instead of Presto's S3FileSystem.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)