[jira] [Created] (SAMZA-2580) Evaluate when to update locality mapping

["Bharath Kumarasubramanian (Jira)" <ji...@apache.org>]

Bharath Kumarasubramanian created SAMZA-2580:
------------------------------------------------

             Summary: Evaluate when to update locality mapping 
                 Key: SAMZA-2580
                 URL: https://issues.apache.org/jira/browse/SAMZA-2580
             Project: Samza
          Issue Type: Improvement
            Reporter: Bharath Kumarasubramanian


Currently, we update the locality mapping as soon as the container starts barring few components (metrics, admins & offset manager). However, the stores get started after we update the locality.

Consider the following scenario
 # A stateful container is running on host A and has a large local state. 
 # The container fails and AM tries to bring it up and fails to get back on the same allocated hosts.
 # After max retries, it chooses a new host (host B) to run the container and the container fails during state restoration.
 # Based on how far the state restoration has progressed it might be optimal to retry to place the container on host A on subsequent attempt rather host B for faster restore & startup time.


The move on the first sight looks theoretically better. However, if it isn't then we should consider moving it outside the container start so simplify wiring locality manager in multiple places. i.e. LocalContainerRunner could write locality and doesn't need to pass it to container.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)