[jira] [Created] (SYSTEMML-1780) Extend resource optimizer for cloud resources

["Matthias Boehm (JIRA)" <ji...@apache.org>]

Matthias Boehm created SYSTEMML-1780:
----------------------------------------

             Summary: Extend resource optimizer for cloud resources
                 Key: SYSTEMML-1780
                 URL: https://issues.apache.org/jira/browse/SYSTEMML-1780
             Project: SystemML
          Issue Type: Task
            Reporter: Matthias Boehm


This task aims to extend the existing resource optimizer, which currently only minimizes time without over-provisioning in bounded YARN cluster by reasoning about CP/MR memory budgets. In detail, we want to include the number of nodes and node types with multiple different optimization criteria: e.g., minimize time under monetary cost constraints, minimize monetary cost under time constraints. 

Note that for iterative algorithms it is impossible to satisfy hard constraints on total execution time without knowing the number of iterations until converge. However, providing these optimization objectives with well-documented heuristics for unknown number of iterations (e.g., fixed constants) would still be very valuable for users who struggle with resource provisioning in cloud environments.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)