You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Cameron Lee (Jira)" <ji...@apache.org> on 2020/02/18 21:41:00 UTC
[jira] [Updated] (SAMZA-2444) JobModel save in
CoordinatorStreamStore resulting flush for each message
[ https://issues.apache.org/jira/browse/SAMZA-2444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cameron Lee updated SAMZA-2444:
-------------------------------
Fix Version/s: 1.4
> JobModel save in CoordinatorStreamStore resulting flush for each message
> ------------------------------------------------------------------------
>
> Key: SAMZA-2444
> URL: https://issues.apache.org/jira/browse/SAMZA-2444
> Project: Samza
> Issue Type: Improvement
> Reporter: Alan Zhang
> Assignee: Alan Zhang
> Priority: Major
> Fix For: 1.4
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> JobModelManager uses CoordinatorStreamStore.put() to save job model information which does flush for each message.
> The flush operation in CoordinatorStreamStore is very heavy, so it may take over a very long time for AM to save the information in a run.
> We should use CoordinatorStreamStore's batch put instead of single put.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)