You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@samza.apache.org by "Shanthoosh Venkataraman (JIRA)" <ji...@apache.org> on 2017/03/20 19:11:41 UTC

[jira] [Created] (SAMZA-1158) Adding monitor to clean up stale local stores of tasks.

Shanthoosh Venkataraman created SAMZA-1158:
----------------------------------------------

             Summary: Adding monitor to clean up stale local stores of tasks.
                 Key: SAMZA-1158
                 URL: https://issues.apache.org/jira/browse/SAMZA-1158
             Project: Samza
          Issue Type: New Feature
    Affects Versions: 0.13.0
            Reporter: Shanthoosh Venkataraman
            Assignee: Shanthoosh Venkataraman


Add samza-rest manager that periodically cleans up the stale local stores of dead jobs/tasks. 

It performs the store deletion in two phases. Initially it deletes the offset file in the local task stores if the following condition is true. ((jobIsNotRunning || preferedHost != nmHost) && offsetFilelastModifiedTime is greater than deleteRetention). 

During the subsequent run, it deletes the local task stores if it does not contain offset file. 

Refer to the design doc of SAMZA-656 (https://issues.apache.org/jira/secure/attachment/12828083/DESIGN-SAMZA-656.pdf) for more details.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)