You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Stephan Ewen (Jira)" <ji...@apache.org> on 2021/03/09 20:28:00 UTC

[jira] [Created] (FLINK-21694) Increase default value of "state.backend.rocksdb.checkpoint.transfer.thread.num"

Stephan Ewen created FLINK-21694:
------------------------------------

             Summary: Increase default value of "state.backend.rocksdb.checkpoint.transfer.thread.num"
                 Key: FLINK-21694
                 URL: https://issues.apache.org/jira/browse/FLINK-21694
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / State Backends
            Reporter: Stephan Ewen
             Fix For: 1.13.0


The default value for the number of threads used to download state artifacts from checkpoint storage should be increased.

The increase should not pose risk of regression, but does in many cases speed up checkpoint recovery significantly.

Something similar was reported in this blog post, item (3).
https://engineering.contentsquare.com/2021/ten-flink-gotchas/

A default value of 8 (eight) sounds like a good default. It should not result in excessive thread explosion, and already speeds up recovery.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)