You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@flink.apache.org by "Jelmer Kuperus (JIRA)" <ji...@apache.org> on 2018/05/04 15:14:00 UTC

[jira] [Created] (FLINK-9297) Make size of JobManager's ioExecutor configurable

Jelmer Kuperus created FLINK-9297:
-------------------------------------

             Summary: Make size of JobManager's ioExecutor configurable
                 Key: FLINK-9297
                 URL: https://issues.apache.org/jira/browse/FLINK-9297
             Project: Flink
          Issue Type: Improvement
          Components: JobManager
    Affects Versions: 1.4.2, 1.4.1, 1.4.0
            Reporter: Jelmer Kuperus


With flink 1.4.0 cleaning of checkpoints is handled on the job manager, the SharedStateRegistry uses the job manager's [io executor|https://github.com/apache/flink/blob/e5ed2fbc4a31808bea8be89b371bbe4269288003/flink-runtime/src/main/scala/org/apache/flink/runtime/jobmanager/JobManager.scala#L1999] as  the executor to schedule disposals of checkpoint state on.  This executor has a fixed size equal to the number of cpu cores. When a lot of small files are created this may not be enough.

It would be good to make this setting configurable. Initializing an executor service with the number of cpu's on the system makes sense for cpu intensive tasks less so for io based workloads

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)