[jira] [Resolved] (SAMZA-2501) Optimizing startpoint manager to not make successive bootstrapMessage calls to coordinator-store

["Bharath Kumarasubramanian (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bharath Kumarasubramanian resolved SAMZA-2501.
----------------------------------------------
    Fix Version/s: 1.5
       Resolution: Fixed

> Optimizing startpoint manager to not make successive bootstrapMessage calls to coordinator-store
> ------------------------------------------------------------------------------------------------
>
>                 Key: SAMZA-2501
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2501
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Rayman
>            Priority: Major
>             Fix For: 1.5
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> Symptom: Currently the startpoint manager queries metastore for task-ssps and for each ssp queries coordinator-store, issuing a re-read on the coordinator store. This causes increased AM startup time and depending on the number of input SSPs to a job (thousands in case of regex), YARN may timeout the AM.
> Cause: Above.
> Fix: This change reduces to number of metastore reads to 2; one for startpoints keyed by ssp and the other for startpoints keyed by ssp+taskname.
> API changes: None
> Upgrade Instructions: None



--
This message was sent by Atlassian Jira
(v8.3.4#803005)