You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Bharath Kumarasubramanian (Jira)" <ji...@apache.org> on 2020/09/23 21:34:00 UTC

[jira] [Created] (SAMZA-2593) Update task callback to store only necessary fields instead of entire message envelope

Bharath Kumarasubramanian created SAMZA-2593:
------------------------------------------------

             Summary: Update task callback to store only necessary fields instead of entire message envelope
                 Key: SAMZA-2593
                 URL: https://issues.apache.org/jira/browse/SAMZA-2593
             Project: Samza
          Issue Type: Improvement
            Reporter: Bharath Kumarasubramanian
            Assignee: Bharath Kumarasubramanian


 TaskCallbackImpl currently stores the entire message envelope. In case of jobs that have 1000s of pending callbacks, this adds up to significant memory pressure. The only use for message envelope is to trace the SSP information and the offset associated with it so that we commit the right offset during our commit cycle.

*Suggested solution*: Modify TaskCallbackImpl to only store SSP and Offset information.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)