[jira] [Resolved] (SAMZA-2439) Remove LocalityManager and container location information from JobModel

["Bharath Kumarasubramanian (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bharath Kumarasubramanian resolved SAMZA-2439.
----------------------------------------------
    Resolution: Fixed

> Remove LocalityManager and container location information from JobModel
> -----------------------------------------------------------------------
>
>                 Key: SAMZA-2439
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2439
>             Project: Samza
>          Issue Type: Improvement
>            Reporter: Prateek Maheshwari
>            Assignee: Bharath Kumarasubramanian
>            Priority: Major
>             Fix For: 1.6
>
>          Time Spent: 7.5h
>  Remaining Estimate: 0h
>
> Goal: Make JobModel immutable by removing locality information (container to host mapping) from the JobModel.
> Motivation: 
> 1. Enables us to move JobModel to public APIs and expose it in JobContext.
> 2. Enables us to cache and serve serialized JobModel from the AM servlet to reduce AM overhead (memory, open connections, num threads) during container startup, esp. for jobs with a large number of containers (See: https://github.com/apache/samza/pull/1241)
> 3. Removes tech debt: models should be immutable, and should not update themselves.
> 4. Removes tech debt: makes current container location a first class concept for container scheduling / placement , and for tools like dashboard, samza-rest, auto-scaling, diagnostics etc.
> Proposal:
> Move locality information (and LocalityManager) out of the JobModel. New JobModel should only contain Config and ContainerModel.
> Add a new AM HTTP endpoint for getting the container location information for external tools.
> Update Samza Dashboard and any other tools that rely on locality information to get it from the new endpoint instead of the JobModel.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)