You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Boris Shkolnik (JIRA)" <ji...@apache.org> on 2019/05/17 19:04:02 UTC
[jira] [Updated] (SAMZA-2169) Preventing task-shuffle after task
mode addition
[ https://issues.apache.org/jira/browse/SAMZA-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Boris Shkolnik updated SAMZA-2169:
----------------------------------
Fix Version/s: 1.2
> Preventing task-shuffle after task mode addition
> ------------------------------------------------
>
> Key: SAMZA-2169
> URL: https://issues.apache.org/jira/browse/SAMZA-2169
> Project: Samza
> Issue Type: Bug
> Reporter: Rayman
> Priority: Major
> Fix For: 1.2
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> After adding task-mode, the first re-deployment of an existing job will not have any existing task-modes to read from the coordinator stream. Due to this the existing task-container mapping of the job does not get read.
> In this case, the grouper will re-assign the tasks to containers in lexicographic order, whereas the previous task-container mapping may have been different than this (based on how the job changed over time – because on topic addition a grouper tries to be as close to existing task-grouping as possible).
> Due to this regrouping, task-container mapping will shuffle, causing some containers to get new tasks, who's store need seeding from Kafka on deployment. This can increase job re-deployment time for the first re-deploy.
> Note that, after the first re-deploy the behavior becomes stable as before.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)