[jira] [Created] (FLINK-14870) Introducing non-nullable assumption for task slot sharing group

["Zhu Zhu (Jira)" <ji...@apache.org>]

Zhu Zhu created FLINK-14870:
-------------------------------

             Summary: Introducing non-nullable assumption for task slot sharing group
                 Key: FLINK-14870
                 URL: https://issues.apache.org/jira/browse/FLINK-14870
             Project: Flink
          Issue Type: Improvement
          Components: Runtime / Coordination
    Affects Versions: 1.10.0
            Reporter: Zhu Zhu


In runtime, a null slot sharing group means the vertex should be run in an individual slot which is not shared with other vertices. 
However, at the moment, there is no way to set the slot sharing group of a vertex to null in production. This is because {{StreamGraphGenerator}} would always assign a non-null slot sharing group to an operator({{StreamNode}}), and that would be the group of the generated {{JobVertex}}. (In the case of {{DataSet}}, all vertices are in the "default" group.)

Moreover, currently users are already able to force an operator to run in an individual slot by specify a different slot sharing group key to that operator.

So looks that supporting null slot sharing groups in runtime does not make much sense. And it's making things more complex, with codes like {{@Nullable}} annotations for slot sharing group, task required resources calculations, and single task slot allocation. 

To make runtime code base and future development simpler, I'd propose to introduce the assumption that task slot sharing group should not be null. 
The detailed proposal is:
1. In {{StreamingJobGraphGenerator}}, we check to ensure the slot sharing group key of a {{StreamNode}} is non-null, so that we always set a non-null {{SlotSharingGroup}} to a JobVertex
2. Remove all the {{@Nullable}} annotations for {{SlotSharingGroup}} and {{SlotSharingGroupID}} in several classes
3. Always use the slot sharing group resources as the physical slot resources for a task
4. Remove {{SchedulerImp#allocateSingleSlot}} and the related code paths

cc [~chesnay] [~gjy] [~trohrmann]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)