You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Bharath Kumarasubramanian (Jira)" <ji...@apache.org> on 2020/09/23 21:35:00 UTC

[jira] [Updated] (SAMZA-2593) Update task callback to store only necessary fields instead of the message envelope

     [ https://issues.apache.org/jira/browse/SAMZA-2593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bharath Kumarasubramanian updated SAMZA-2593:
---------------------------------------------
    Summary: Update task callback to store only necessary fields instead of the message envelope  (was: Update task callback to store only necessary fields instead of entire message envelope)

> Update task callback to store only necessary fields instead of the message envelope
> -----------------------------------------------------------------------------------
>
>                 Key: SAMZA-2593
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2593
>             Project: Samza
>          Issue Type: Improvement
>            Reporter: Bharath Kumarasubramanian
>            Assignee: Bharath Kumarasubramanian
>            Priority: Major
>
>  TaskCallbackImpl currently stores the entire message envelope. In case of jobs that have 1000s of pending callbacks, this adds up to significant memory pressure. The only use for message envelope is to trace the SSP information and the offset associated with it so that we commit the right offset during our commit cycle.
> *Suggested solution*: Modify TaskCallbackImpl to only store SSP and Offset information.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)