You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Weihua Hu (Jira)" <ji...@apache.org> on 2022/11/03 12:52:00 UTC
[jira] [Updated] (FLINK-29870) split ResourceNotEnoughNotifier from ResourceActions
[ https://issues.apache.org/jira/browse/FLINK-29870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Weihua Hu updated FLINK-29870:
------------------------------
Description:
SlotManager determines whether a resource can be allocated by the return value of ResourceActions.allocateResource. It can reduce unnecessary waiting by quickly marking the slotRequest as a failure if the resource cannot be allocated.
The new declareResourceNeed function will not return whether resources could be allocated. This will cause useless waiting for SlotRequests.
Currently, ResourceActions does only two kinds of things.
allocate/release resource by ResourceManager, this only supported by ActiveResourceManager
notify resource not enough to JobMaster
So we can split ResourceNotEnoughNotifier from ResourceActions, and make ResourceActions only works for ActiveResourceManager
ResourceNotEnoughNotifier only send notifyNotEnoughResourcesAvailable to JobMaster
ResourceActions will deal with declareResourcesNeeded
For StandaloneResourceManager, The SlotManager will not have one ResourceAllocator, so it can fail the SlotRequests quickly.
was:
SlotManager determines whether a resource can be allocated by the return value of ResourceActions.allocateResource. It can reduce unnecessary waiting by quickly marking the slotRequest as a failure if the resource cannot be allocated.
The new declareResourceNeed function will not return whether resources could be allocated. This will cause useless waiting for SlotRequests.
Currently, ResourceActions does only two kinds of things.
allocate/release resource by ResourceManager, this only supported by ActiveResourceManager
notify resource not enough to JobMaster
So we can split ResourceActions to ResourceAllocator(Only works in ActiveResourceManager) and NoEnoughResourceNotifier.
NoEnoughResourceNotifier only send notifyNotEnoughResourcesAvailable to JobMaster
ResourceAllocator will deal with declareResourcesNeeded
For StandaloneResourceManager, The SlotManager will not have one ResourceAllocator, so it can fail the SlotRequests quickly.
Summary: split ResourceNotEnoughNotifier from ResourceActions (was: Split ResourceActions to ResourceAllocator and ResourceAllocationNotifier)
> split ResourceNotEnoughNotifier from ResourceActions
> ----------------------------------------------------
>
> Key: FLINK-29870
> URL: https://issues.apache.org/jira/browse/FLINK-29870
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Affects Versions: 1.16.0
> Reporter: Weihua Hu
> Priority: Major
> Fix For: 1.17.0
>
>
> SlotManager determines whether a resource can be allocated by the return value of ResourceActions.allocateResource. It can reduce unnecessary waiting by quickly marking the slotRequest as a failure if the resource cannot be allocated.
> The new declareResourceNeed function will not return whether resources could be allocated. This will cause useless waiting for SlotRequests.
> Currently, ResourceActions does only two kinds of things.
> allocate/release resource by ResourceManager, this only supported by ActiveResourceManager
> notify resource not enough to JobMaster
> So we can split ResourceNotEnoughNotifier from ResourceActions, and make ResourceActions only works for ActiveResourceManager
> ResourceNotEnoughNotifier only send notifyNotEnoughResourcesAvailable to JobMaster
> ResourceActions will deal with declareResourcesNeeded
> For StandaloneResourceManager, The SlotManager will not have one ResourceAllocator, so it can fail the SlotRequests quickly.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)