You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Shanthoosh Venkataraman (JIRA)" <ji...@apache.org> on 2019/05/17 19:27:00 UTC

[jira] [Closed] (SAMZA-2161) Move ChangelogPartitionManager and CoordinatorStream ConfigReader to MetadataStore

     [ https://issues.apache.org/jira/browse/SAMZA-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shanthoosh Venkataraman closed SAMZA-2161.
------------------------------------------

> Move ChangelogPartitionManager and CoordinatorStream ConfigReader to MetadataStore
> ----------------------------------------------------------------------------------
>
>                 Key: SAMZA-2161
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2161
>             Project: Samza
>          Issue Type: Improvement
>            Reporter: Shanthoosh Venkataraman
>            Assignee: Shanthoosh Venkataraman
>            Priority: Major
>             Fix For: 1.2
>
>          Time Spent: 4.5h
>  Remaining Estimate: 0h
>
> Currently the metadata of a samza job is stored into a kafka topic named coordinator stream.
> In samza-yarn ApplicationMaster, the same coordinator stream is read twice as a part of the startup sequence. This duplicate read unnecessarily prolongs the startup time of the application master and makes the container allocation take longer than usual. This inadvertently incurs a substantial increase in input stream processing delay depending upon the size of the coordinator stream. 
> To mitigate this problem, the two util classes in samza viz `ChangelogPartitionManager`, `Config` should be moved to use the MetadataStore abstraction. This ticket tracks the work involved in the migration. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)