[jira] [Updated] (TEZ-932) Add a type aware Initial memory distributor

["Siddharth Seth (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/TEZ-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Siddharth Seth updated TEZ-932:
-------------------------------

    Attachment: TEZ-932.1.txt

Patch does the following
- Pulls out the InitialAllocator interfaces and makes them pluggable
- Adds an allocator which does weighted scaling based on the Runtime-library components being used. As an example this can be configured to say ratio of mem available for ShuffledUnorderedKVInput to ShuffledMergedKVInput is 1:20.

[~hitesh], could you please take a look.

> Add a type aware Initial memory distributor
> -------------------------------------------
>
>                 Key: TEZ-932
>                 URL: https://issues.apache.org/jira/browse/TEZ-932
>             Project: Apache Tez
>          Issue Type: Improvement
>            Reporter: Siddharth Seth
>            Assignee: Siddharth Seth
>         Attachments: TEZ-932.1.txt
>
>
> The current memory scaler has not idea about the requirements of the underlying components (broadcast, shfufle etc). This jira is to add a scaler which can take the types into consideration.



--
This message was sent by Atlassian JIRA
(v6.2#6252)