[jira] [Commented] (AURORA-1415) De-generalize resource handling in Scheduler

["Maxim Khutornenko (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AURORA-1415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14654067#comment-14654067 ] 

Maxim Khutornenko commented on AURORA-1415:
-------------------------------------------

h3. Problem
As it stands today, scheduler resource management is unable to support custom resource attributes/traits defined in Mesos. Scheduler treats host resources as homogenous scalar vectors (CPU, RAM, DISK) or ranges (PORTS) and rolls them up without considering their unique attributes (e.g. [role|https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L494] or [revokable|https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L560]). This makes it impossible to support certain Mesos features that rely on resource customization ([framework role|AURORA-1109], [revokable offers|AURORA-1343], [dynamic reservation|https://docs.google.com/document/d/1e3j69pfBgtc8xM00DhcuiMl6ImkEB5na0TzOMyzrg8A/edit]). Scheduler resource management needs to recognize and expose Mesos resource attributes. This will require de-anonymizing resources and preventing their automatic rollup based on vector type. 

h3. Current Implementation
The bulk of resource management in scheduler is represented by the following classes:
- {{Resources}} - main resource container and handler. Converts from/to Mesos protobuf resources to/From internal representation (scalar/range values for cpu, ram, disk and ports). 
- {{ResourceSlot}} - wrapper over {{Resources}} providing additional logic for executor overhead and utility functions.
- {{ResourceAggregate}} - internal thrift object intended to communicate aggregated cpu/ram/disk values with client/UI.
- {{ResourceAggregates}} - static library helpers for dealing with {{ResourceAggregate}} instances.

h3. Proposal
Mesos {{Resource}} protobuf [struct|https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto#L488] is already quite rich with attributes to mirror it on the scheduler side with a correspondent POJO. A more resilient long term approach may be holding on to the original Mesos protobuf structs within our internal {{Resources}} object before converting them into a generalized aggregated representation. Below is a proposed set of changes intending to de-generalize resource management and enable filtering/querying by arbitrary resource attributes:
- {{Resources}} will store a list of original resource vectors unchanged. E.g.: {{Iterable<Resource> mesosResources;}}.
- {{Resources}} will support a new {{Resources filter(Predicate<Resource> predicate);}} to filter resources by a given attribute.
- {{ResourceSlot}} will be repurposed to serve as an anonymous (aggregated resource representation) to be returned by the new {{ResourceSlot aggregate();}} method in {{Resources}} any time an aggregation or transformation is needed.
- Static transformation methods (like {{subtract}}, {{divide}} and etc.) will become {{ResourceSlot}} instance methods. 
- {{ResourceAggregates}} will be merged with {{ResourceSlot}} to convert to/from {{ResourceAggregate}} when needed.

h4. Usage Examples
h6. Getting available revocable resources from an offer
{noformat} 
public static final Predicate<Resource> CPU = e -> e.getName().equals("cpu");
public static final Predicate<Resource> REVOCABLE = e -> CPU.negate() || CPU.and(e.getRevocable() != null);
...
ResourceSlot unused = Resources.from(offer).filter(REVOCABLE).aggregate();
{noformat}

h6. Getting available revocable resources from an offer for a given framework role
{noformat}
public static final Predicate<Resource> ROLE = e -> e.getRole().equals("Aurora");
...
ResourceSlot roleRevocable = Resources.from(offer).filter(ROLE.and(REVOCABLE)).aggregate();
{noformat}

h6. Creating Mesos resources for TaskInfo with executor overhead (and epsilon, see {{MesosTaskFactory}} for real life example)
{noformat}
List<Resource> resources = Resources.from(ResourceSlot.from(task).add(executorOverhead).subtract(RESOURCES_EPSILON)).toResourceList();
{noformat}


> De-generalize resource handling in Scheduler
> --------------------------------------------
>
>                 Key: AURORA-1415
>                 URL: https://issues.apache.org/jira/browse/AURORA-1415
>             Project: Aurora
>          Issue Type: Task
>          Components: Scheduler
>            Reporter: Maxim Khutornenko
>            Assignee: Maxim Khutornenko
>
> From design doc: 
> {quote}
> To handle revocable resources correctly Aurora needs to de-generalize and simplify its internal resource representation. The new resource vector should be capable of aggregating resources by revocable flag. This will require refactoring existing resource handling (including  AURORA-105), which will also help to support Mesos framework role in future.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)