You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "zoucao (Jira)" <ji...@apache.org> on 2022/06/06 10:25:00 UTC

[jira] [Updated] (FLINK-27911) Set HDFS LEASE_TIMEOUT as user-configurable

     [ https://issues.apache.org/jira/browse/FLINK-27911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

zoucao updated FLINK-27911:
---------------------------
    Affects Version/s: 1.14.4

> Set HDFS LEASE_TIMEOUT as user-configurable
> -------------------------------------------
>
>                 Key: FLINK-27911
>                 URL: https://issues.apache.org/jira/browse/FLINK-27911
>             Project: Flink
>          Issue Type: Improvement
>    Affects Versions: 1.12.2, 1.14.4
>            Reporter: zoucao
>            Priority: Major
>         Attachments: exceptions
>
>
> The HDFS LEASE_TIMEOUT in *HadoopRecoverableFsDataOutputStream* is set to 100_000 millis, In some cases, 100 seconds is not enough. In our company, when using StreamingFileSink to write records to HDFS and the parallelism is set to 512 or larger, the following exceptions occur frequently. I think we can update the parameter LEASE_TIMEOUT as user-configurable.



--
This message was sent by Atlassian Jira
(v8.20.7#820007)