You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Yixing Zhang (Jira)" <ji...@apache.org> on 2021/02/02 19:53:00 UTC
[jira] [Commented] (SAMZA-2439) Remove LocalityManager and
container location information from JobModel
[ https://issues.apache.org/jira/browse/SAMZA-2439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277428#comment-17277428 ]
Yixing Zhang commented on SAMZA-2439:
-------------------------------------
Should we close this ticket? [~bharathkk]
This pr has been checked in: > f7f9f3c7 - (Wed Aug 26 19:39:32 2020 -0700) SAMZA-2439: Remove LocalityManager and container location information from JobModel (#1421) - mynameborat
> Remove LocalityManager and container location information from JobModel
> -----------------------------------------------------------------------
>
> Key: SAMZA-2439
> URL: https://issues.apache.org/jira/browse/SAMZA-2439
> Project: Samza
> Issue Type: Improvement
> Reporter: Prateek Maheshwari
> Assignee: Bharath Kumarasubramanian
> Priority: Major
> Fix For: 1.6
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Goal: Make JobModel immutable by removing locality information (container to host mapping) from the JobModel.
> Motivation:
> 1. Enables us to move JobModel to public APIs and expose it in JobContext.
> 2. Enables us to cache and serve serialized JobModel from the AM servlet to reduce AM overhead (memory, open connections, num threads) during container startup, esp. for jobs with a large number of containers (See: https://github.com/apache/samza/pull/1241)
> 3. Removes tech debt: models should be immutable, and should not update themselves.
> 4. Removes tech debt: makes current container location a first class concept for container scheduling / placement , and for tools like dashboard, samza-rest, auto-scaling, diagnostics etc.
> Proposal:
> Move locality information (and LocalityManager) out of the JobModel. New JobModel should only contain Config and ContainerModel.
> Add a new AM HTTP endpoint for getting the container location information for external tools.
> Update Samza Dashboard and any other tools that rely on locality information to get it from the new endpoint instead of the JobModel.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)