[jira] [Commented] (SAMZA-2689) [Elasticity] introduce ElasticJobModel which has SystemStreamPartitionKeyHash

["Lakshmi Manasa Gaduputi (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SAMZA-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17494851#comment-17494851 ] 

Lakshmi Manasa Gaduputi commented on SAMZA-2689:
------------------------------------------------

[https://github.com/apache/samza/pull/1580] merged

> [Elasticity] introduce ElasticJobModel which has SystemStreamPartitionKeyHash
> -----------------------------------------------------------------------------
>
>                 Key: SAMZA-2689
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2689
>             Project: Samza
>          Issue Type: Sub-task
>            Reporter: Lakshmi Manasa Gaduputi
>            Assignee: Lakshmi Manasa Gaduputi
>            Priority: Major
>          Time Spent: 4h 40m
>  Remaining Estimate: 0h
>
> # For a job with elasticity factor = X,  each input SystemStreamPartition is split into X parts aka SystemStreamPartitionKeyHashes. ElasticJobModel has elastic task models where each original task ( in the original JobModel) now has X elastic tasks each of which consumes an SSPKh pertaining to the SSP of the original task.
>  # Create an ElasticJobModel and thus assign elastic tasks to containers. To achieve this, the groupers which group the input SSPs into task models, need to be updated. 
>  # ElasticJobModel created in the earlier sub-task creates X elastic tasks for each original task where elasticity.factor=X. Each elastic task consumes an SSPKh = 1/X th of the input SSP. This task is to ensure each elastic task actually processes only the messages within that SSPKh.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)