[jira] [Created] (FLINK-31366) Exception is thrown when s3a and s3p are used together

["Hangxiang Yu (Jira)" <ji...@apache.org>]

Hangxiang Yu created FLINK-31366:
------------------------------------

             Summary: Exception is thrown when s3a and s3p are used together
                 Key: FLINK-31366
                 URL: https://issues.apache.org/jira/browse/FLINK-31366
             Project: Flink
          Issue Type: Bug
          Components: FileSystems
            Reporter: Hangxiang Yu
            Assignee: Hangxiang Yu


h3. Exception

When s3a and s3p plugins exist at the same time, an exception will be thrown as below:
{code:java}
Caused by:java.lang.NoSuchMethodError: org.apache.flink.fs.s3.common.AbstractS3FileSystemFactory.<init> (Ljava/lang/String;Lorg/apache/flink/fs/s3presto/common/HadoopConfigLoader;)
{code}
h3. Why

In the construction method of AbstractS3FileSystemFactory, s3a shades HadoopConfigLoader into org.apache.flink.fs.s3hadoop.common.HadoopConfigLoader, and s3p shades HadoopConfigLoader into org. apache.flink.fs.s3presto.common.HadoopConfigLoader when package.

(see shade-plugin)

 

But the AbstractS3FileSystemFactory class will only be loaded once when loading even if there are two plugins.

So it may first uses s3a plugin to load AbstractS3FileSystemFactory, and the construction method is loaded into AbstractS3FileSystemFactory(String name, org.apache. flink.fs.s3hadoop.common.HadoopConfigLoader hadoopConfigLoader), at this time s3p will be abnormal when it is constructed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)