[jira] [Updated] (FLINK-13602) S3 filesystems effectively do not support credential providers

["Chesnay Schepler (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-13602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Chesnay Schepler updated FLINK-13602:
-------------------------------------
    Summary: S3 filesystems effectively do not support credential providers  (was: Presto S3 filesystem effectively does not support credential providers)

> S3 filesystems effectively do not support credential providers
> --------------------------------------------------------------
>
>                 Key: FLINK-13602
>                 URL: https://issues.apache.org/jira/browse/FLINK-13602
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / FileSystem
>    Affects Versions: 1.9.0
>            Reporter: Chesnay Schepler
>            Priority: Major
>
> To provide credentials to S3 users may configure a credentials provider. For providers from amazon (which are relocated) we allow users to configure the original class name, and relocate it manually in the S3 filesystem factories.
> However, none of the amazon provided credential providers can be used with the Presto filesystem, since it _additionally_ requires them to have a constructor accepting a hadoop configuration. (https://prestodb.github.io/docs/current/connector/hive.html#amazon-s3-configuration)
> {{hadoop-aws}} _does_ include a number of credential providers that have this constructor, however these use configuration keys that aren't mirrored from the flink config. (they expect {{fs.s3a}} as a key-prefix), not to mention that users would have to configure the relocated class (since the S3 factory only manually relocates amazon classes).
> Finally, a custom implementation of the credentials provider can effectively be ruled out since they too would have to be implemented against the relocated amazon/hadoop classes, which we can't really expect users to do.
> In summary, amazon providers aren't working since they don't have a constructor that presto requires, hadoop providers don't work since we don't mirror the required configuration keys, and custom providers are unreasonable as they'd have to be implemented against relocated classes.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)