[jira] [Commented] (FLINK-31366) Exception is thrown when s3a and s3p are used together

["Yun Tang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-31366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17697783#comment-17697783 ] 

Yun Tang commented on FLINK-31366:
----------------------------------

I wonder will it break the backward compatibility with your current implementation?

> Exception is thrown when s3a and s3p are used together
> ------------------------------------------------------
>
>                 Key: FLINK-31366
>                 URL: https://issues.apache.org/jira/browse/FLINK-31366
>             Project: Flink
>          Issue Type: Bug
>          Components: FileSystems
>            Reporter: Hangxiang Yu
>            Assignee: Hangxiang Yu
>            Priority: Major
>              Labels: pull-request-available
>
> h3. Exception
> When s3a and s3p plugins exist at the same time, an exception will be thrown as below:
> {code:java}
> Caused by:java.lang.NoSuchMethodError: org.apache.flink.fs.s3.common.AbstractS3FileSystemFactory.<init> (Ljava/lang/String;Lorg/apache/flink/fs/s3presto/common/HadoopConfigLoader;)
> {code}
> h3. Why
> In the construction method of AbstractS3FileSystemFactory, s3a shades HadoopConfigLoader into org.apache.flink.fs.s3hadoop.common.HadoopConfigLoader, and s3p shades HadoopConfigLoader into org. apache.flink.fs.s3presto.common.HadoopConfigLoader when package.
> (see shade-plugin)
>  
> But the AbstractS3FileSystemFactory class will only be loaded once when loading even if there are two plugins.
> So it may first uses s3a plugin to load AbstractS3FileSystemFactory, and the construction method is loaded into AbstractS3FileSystemFactory(String name, org.apache. flink.fs.s3hadoop.common.HadoopConfigLoader hadoopConfigLoader), at this time s3p will be abnormal when it is constructed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)