[jira] [Closed] (FLINK-17865) Increase default size of `state.backend.fs.memory-threshold`

["Yu Li (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/FLINK-17865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yu Li closed FLINK-17865.
-------------------------
    Release Note: After FLINK-17865 the the default value of `state.backend.fs.memory-threshold` has been increased from 1K to 20K to prevent too many small files created on remote FS for small states. Notice: jobs with large parallelism on source or stateful operators may have "JM OOM" or "RPC message exceeding maximum frame size" problem with this change, and please manually set the configuration back to 1K if you encounter such issues. 
      Resolution: Done

Merged in
master via a835f31a3a78f34b4a80f9e634b34c6a6681a482
release-1.11 via 6f9042b5f253a077af3fce918c01475f1274aef1

> Increase default size of `state.backend.fs.memory-threshold`
> ------------------------------------------------------------
>
>                 Key: FLINK-17865
>                 URL: https://issues.apache.org/jira/browse/FLINK-17865
>             Project: Flink
>          Issue Type: Improvement
>          Components: Runtime / State Backends
>    Affects Versions: 1.11.0
>            Reporter: Yu Li
>            Assignee: Yun Tang
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 1.11.0
>
>
> As discussed in [ML thread|https://s.apache.org/us0en], we decided to increase the default value of `state.backend.fs.memory-threshold` from 1K to *20K* to prevent too many small files created on remote FS.
> We need to add an explicit release note to notify our users about this change, especially for those having jobs with large parallelism on source or stateful operators.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)