You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Zhu Zhu (Jira)" <ji...@apache.org> on 2020/09/10 09:00:00 UTC
[jira] [Closed] (FLINK-14870) Remove nullable assumption of task
slot sharing group
[ https://issues.apache.org/jira/browse/FLINK-14870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Zhu Zhu closed FLINK-14870.
---------------------------
Resolution: Fixed
> Remove nullable assumption of task slot sharing group
> -----------------------------------------------------
>
> Key: FLINK-14870
> URL: https://issues.apache.org/jira/browse/FLINK-14870
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.10.0
> Reporter: Zhu Zhu
> Assignee: Zhu Zhu
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.12.0
>
>
> In runtime, a null slot sharing group means the vertex should be run in an individual slot which is not shared with other vertices.
> However, at the moment, there is no way to set the slot sharing group of a vertex to null in production. This is because {{StreamGraphGenerator}} would always assign a non-null slot sharing group to an operator({{StreamNode}}), and that would be the group of the generated {{JobVertex}}. (In the case of {{DataSet}}, all vertices are in the "default" group.)
> Moreover, currently users are already able to force an operator to run in an individual slot by specify a different slot sharing group key to that operator.
> So looks that supporting null slot sharing groups in runtime does not make much sense. And it's making things more complex, with codes like {{@Nullable}} annotations for slot sharing group, task required resources calculations, and single task slot allocation.
> To make runtime code base and future development simpler, I'd propose to introduce the assumption that task slot sharing group should not be null.
> The detailed proposal is:
> 1. In {{StreamingJobGraphGenerator}}, we check to ensure the slot sharing group key of a {{StreamNode}} is non-null, so that we always set a non-null {{SlotSharingGroup}} to a JobVertex
> 2. Remove all the {{@Nullable}} annotations for {{SlotSharingGroup}} and {{SlotSharingGroupID}} in several classes
> -3. Always use the slot sharing group resources as the physical slot resources for a task-
> 4. Remove {{SchedulerImp#allocateSingleSlot}} and the related code paths
> cc [~chesnay] [~gjy] [~trohrmann]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)