You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Lakshmi Manasa Gaduputi (Jira)" <ji...@apache.org> on 2022/02/01 17:21:00 UTC
[jira] [Updated] (SAMZA-2689) [Elasticity] introduce ElasticJobModel which has SystemStreamPartitionKeyHash
[ https://issues.apache.org/jira/browse/SAMZA-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lakshmi Manasa Gaduputi updated SAMZA-2689:
-------------------------------------------
Description:
# For a job with elasticity factor = X, each input SystemStreamPartition is split into X parts aka SystemStreamPartitionKeyHashes. ElasticJobModel has elastic task models where each original task ( in the original JobModel) now has X elastic tasks each of which consumes an SSPKh pertaining to the SSP of the original task.
# Create an ElasticJobModel and thus assign elastic tasks to containers. To achieve this, the groupers which group the input SSPs into task models, need to be updated.
# ElasticJobModel created in the earlier sub-task creates X elastic tasks for each original task where elasticity.factor=X. Each elastic task consumes an SSPKh = 1/X th of the input SSP. This task is to ensure each elastic task actually processes only the messages within that SSPKh.
was:# For a job with elasticity factor = X, each input SystemStreamPartition is split into X parts aka SystemStreamPartitionKeyHashes. ElasticJobModel has elastic task models where each original task ( in the original JobModel) now has X elastic tasks each of which consumes an SSPKh pertaining to the SSP of the original task.
> [Elasticity] introduce ElasticJobModel which has SystemStreamPartitionKeyHash
> -----------------------------------------------------------------------------
>
> Key: SAMZA-2689
> URL: https://issues.apache.org/jira/browse/SAMZA-2689
> Project: Samza
> Issue Type: Sub-task
> Reporter: Lakshmi Manasa Gaduputi
> Assignee: Lakshmi Manasa Gaduputi
> Priority: Major
>
> # For a job with elasticity factor = X, each input SystemStreamPartition is split into X parts aka SystemStreamPartitionKeyHashes. ElasticJobModel has elastic task models where each original task ( in the original JobModel) now has X elastic tasks each of which consumes an SSPKh pertaining to the SSP of the original task.
> # Create an ElasticJobModel and thus assign elastic tasks to containers. To achieve this, the groupers which group the input SSPs into task models, need to be updated.
> # ElasticJobModel created in the earlier sub-task creates X elastic tasks for each original task where elasticity.factor=X. Each elastic task consumes an SSPKh = 1/X th of the input SSP. This task is to ensure each elastic task actually processes only the messages within that SSPKh.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)