You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xintong Song (Jira)" <ji...@apache.org> on 2020/01/15 12:06:00 UTC
[jira] [Comment Edited] (FLINK-12122) Spread out tasks evenly
across all available registered TaskManagers
[ https://issues.apache.org/jira/browse/FLINK-12122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015892#comment-17015892 ]
Xintong Song edited comment on FLINK-12122 at 1/15/20 12:05 PM:
----------------------------------------------------------------
[~trohrmann],
Do you think the approach mentioned by your [comment|https://issues.apache.org/jira/browse/FLINK-15527?focusedCommentId=17013086&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17013086] in FLINK-15527 could help on this issue?
If we have something like the min/max number of task executors, and the min number of task executors are started immediately when the cluster is started, then it's similar to a standalone cluster and the current evenly-spread-out-slots-strategy applies.
We might also need to wait for the min number of TM registered to accept JM registrations and slot requests, to make sure TMs register before slot requests arrive.
was (Author: xintongsong):
[~trohrmann],
Do you think the approach we mentioned by your [comment|https://issues.apache.org/jira/browse/FLINK-15527?focusedCommentId=17013086&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17013086] in FLINK-15527 could help on this issue?
If we have something like the min/max number of task executors, and the min number of task executors are started immediately when the cluster is started, then it's similar to a standalone cluster and the current evenly-spread-out-slots-strategy applies.
We might also need to wait for the min number of TM registered to accept JM registrations and slot requests, to make sure TMs register before slot requests arrive.
> Spread out tasks evenly across all available registered TaskManagers
> --------------------------------------------------------------------
>
> Key: FLINK-12122
> URL: https://issues.apache.org/jira/browse/FLINK-12122
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.6.4, 1.7.2, 1.8.0
> Reporter: Till Rohrmann
> Assignee: Till Rohrmann
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.9.2, 1.10.0
>
> Attachments: image-2019-05-21-12-28-29-538.png, image-2019-05-21-13-02-50-251.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> With Flip-6, we changed the default behaviour how slots are assigned to {{TaskManages}}. Instead of evenly spreading it out over all registered {{TaskManagers}}, we randomly pick slots from {{TaskManagers}} with a tendency to first fill up a TM before using another one. This is a regression wrt the pre Flip-6 code.
> I suggest to change the behaviour so that we try to evenly distribute slots across all available {{TaskManagers}} by considering how many of their slots are already allocated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)