[jira] [Updated] (MESOS-3896) Add accounting for reservation slack in the allocator.

["Klaus Ma (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MESOS-3896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Klaus Ma updated MESOS-3896:
----------------------------
    Assignee:     (was: Klaus Ma)

> Add accounting for reservation slack in the allocator.
> ------------------------------------------------------
>
>                 Key: MESOS-3896
>                 URL: https://issues.apache.org/jira/browse/MESOS-3896
>             Project: Mesos
>          Issue Type: Bug
>            Reporter: Artem Harutyunyan
>              Labels: mesosphere
>
> MESOS-XXX: Optimsistic accounter
> {code}
>     class HierarchicalAllocatorProcess 
>     {
>       struct Slave
>       {
>         ...
>         struct Optimistic 
>         {
>           Resources total; // The total allocation slack resources
>           Resources allocated; // The allocated allocation slack resources
>         };
>     
>         Optimistic optimistic;
>       };
>     }
> {code}
> MESOS-4146: flatten & allocationSlack for Optimistic Offer
> {code}
>     class Resources
>     {
>         // Returns a Resources object with the same amount of each resource
>         // type as these Resources, but with all Resource objects marked as
>         // the specified `RevocableInfo::Type`; the other attribute is not
>         // affected.
>         Resources flatten(Resource::RevocableInfo::Type type);
>         // Return a Resources object that:
>         //   - if role is given, the resources did not include role's reserved
>         //     resources.
>         //   - the resources's revocable type is `ALLOCATION_SLACK`
>         //   - the role of resources is set to "*"
>         Resources allocationSlack(Option<string> role = None());
>     }
> {code}
> MESOS-XXX: Allocate the allocation_slack resources to framework
> {code}
>     void HierarchicalAllocatorProcess::allocate(
>         const hashset<SlaveID>& slaveIds_)
>     {
>       foreach slave; foreach role; foreach framework
>       {
>         Resource optimistic;
>         if (framework.revocable) {
>           Resources total = slaves[slaveId].optimistic.total.allocationSlack(role);
>           optimistic = total - slaves[slaveId].optimistic.allocated;
>         }
>         ...
>         offerable[frameworkId][slaveId] += resources + optimistic;
>         ...
>         slaves[slaveId].optimistic.allocated += optimistic;
>       }
>     }
> {code}
>   
> Here's some consideration about `ALLOCATION_SLACK`:
> 1. 'Old' resources (available/total) did not include ALLOCATION_SLACK
> 2. After `Quota`, `remainingClusterResources.contains` should not check ALLOCATION_SLACK; if there no enough resources,  master can still offer ALLOCATION_SALCK resources.
> 3. In sorter, it'll not include ALLOCATION_SLACK; as those resources are borrowed from other role/framework
> 4. If either normal resources or ALLOCATION_SLACK resources are allocable/!filtered, it can be offered to framework
> 5. Currently, allocator will assign all ALLOCATION_SALCK resources in slave to one framework
> MESOS-XXX: Update ALLOCATION_SLACK for dynamic reservation (updateAllocation)
> {code}
>     void HierarchicalAllocatorProcess::updateAllocation(
>         const FrameworkID& frameworkId,
>         const SlaveID& slaveId,
>         const vector<Offer::Operation>& operations)
>     {
>         ...
>         Try<Resources> updatedOptimistic =
>             slaves[slaveId].optimistic.total.apply(operations);
>         CHECK_SOME(updatedTotal);
>         slaves[slaveId].optimistic.total =
>             updatedOptimistic.get().stateless().reserved().flatten(ALLOCATION_SLACK);
>         ...
>     }
> {code}
>     
> MESOS-XXX: Add ALLOCATION_SLACK when slaver register/re-register (addSlave)
> {code}
>     void HierarchicalAllocatorProcess::addSlave(
>         const SlaveID& slaveId,
>         const SlaveInfo& slaveInfo,
>         const Option<Unavailability>& unavailability,
>         const Resources& total,
>         const hashmap<FrameworkID, Resources>& used)
>     {
>       ...
>       slaves[slaveId].optimistic.total =
>           total.stateless().reserved().flatten(ALLOCATION_SLACK);
>       ...
>     }
> {code}
>   
> No need to handle `removeSlave`, it'll all related info from `slaves` including `optimistic`.
> MESOS-XXX: return resources to allocator (recoverResources)
> {code}
>     void HierarchicalAllocatorProcess::recoverResources(
>         const FrameworkID& frameworkId,
>         const SlaveID& slaveId,
>         const Resources& resources,
>         const Option<Filters>& filters)
>     {
>       if (slaves.contains(slaveId))
>       {
>         ...
>         slaves[slaveId].optimistic.allocated -= resources.allocationSlack();
>         ...
>       }
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)