[jira] [Assigned] (FLINK-15777) Truncate not working with plugins (as expected)

["Kostas Kloudas (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-15777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kostas Kloudas reassigned FLINK-15777:
--------------------------------------

    Assignee: Arvid Heise

> Truncate not working with plugins (as expected)
> -----------------------------------------------
>
>                 Key: FLINK-15777
>                 URL: https://issues.apache.org/jira/browse/FLINK-15777
>             Project: Flink
>          Issue Type: Bug
>          Components: FileSystems
>    Affects Versions: 1.10.0, 1.11.0
>            Reporter: Arvid Heise
>            Assignee: Arvid Heise
>            Priority: Critical
>              Labels: pull-request-available
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> FLINK-14170 introduced Hadoop version check to support older versions with limited functionality. However, the fix checks the Hadoop version of the system classloader and not of the plugin (of the actual used filesystem).
> The [version checks|https://github.com/apache/flink/blob/a607bd9dbdda7d9925d6c351ba88d82edce2c571/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopRecoverableFsDataOutputStream.java#L197] will fail when there is a Hadoop version in the classpath of Flink, independent of the bundled version of the filesystem.
> On a recent EMR setup with lingering Hadoop 2.6 libraries, this mean that you cannot write to S3 with StreamingFileSink.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)