You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Roman Khachatryan (Jira)" <ji...@apache.org> on 2024/05/31 18:24:00 UTC

[jira] [Created] (FLINK-35501) Use common thread pools when transferring RocksDB state files

Roman Khachatryan created FLINK-35501:
-----------------------------------------

             Summary: Use common thread pools when transferring RocksDB state files
                 Key: FLINK-35501
                 URL: https://issues.apache.org/jira/browse/FLINK-35501
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / State Backends
    Affects Versions: 1.20.0
            Reporter: Roman Khachatryan
            Assignee: Roman Khachatryan
             Fix For: 1.20.0


Currently, each RocksDB state backend creates an executor backed by a thread pool.

This makes it difficult to control the total number of threads per TM because it might have at least one task per slot and theoretically, many state backends per task (because of chaining).

Additionally, using a common thread pool allows to indirectly control the load on the underlying DFS (e.g. the total number of requests to S3 from a TM).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)