[jira] [Commented] (FLINK-38350) Update Hadoop Library to 3.4.2

["Martijn Visser (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-38350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18020267#comment-18020267 ] 

Martijn Visser commented on FLINK-38350:
----------------------------------------

[~pashley] I would love for us to upgrade, we tried but also failed because the Hadoop upgrades have breaking changes that would require an extensive rewrite. See the PR at https://github.com/apache/flink/pull/23844#issuecomment-1871212810 and the related ticket https://issues.apache.org/jira/browse/FLINK-33584

> Update Hadoop Library to 3.4.2
> ------------------------------
>
>                 Key: FLINK-38350
>                 URL: https://issues.apache.org/jira/browse/FLINK-38350
>             Project: Flink
>          Issue Type: Improvement
>          Components: FileSystems
>    Affects Versions: 2.0-preview, 1.20.2
>            Reporter: Paul Ashley
>            Priority: Major
>             Fix For: 1.20.3
>
>   Original Estimate: 168h
>  Remaining Estimate: 168h
>
> Hadoop 3.4.2 which includes support for AWS conditional writes in https://issues.apache.org/jira/browse/HADOOP-19256
>  
> It would be valuable for Flink to include this, as it would allow S3 based table sinks to be able to guarantee write integrity without relying on other concurrency controls (or accepting data loss).  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)