[jira] [Updated] (SAMZA-2664) Checkpoint validation enabled needs to be false forever if grouper factory was changed.

["Lakshmi Manasa Gaduputi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-2664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lakshmi Manasa Gaduputi updated SAMZA-2664:
-------------------------------------------
    Description: 
TO change the grouper factory of the job, checkpoint validation should be disabled (using the config = ). This ensures the new checkpoints can be written with the new factory and job can use the new checkpoints. However, since the factory is part of the checkpoint key, the older checkpoints are never cleaned up. This leads to job reading those old checkpoints on every deploy and failing checkpoint validation. to get job running, checkpoint validation needs to be disabled forever.

Keeping checkpoint validation disabled forever is risky, as that would ignore the exceptions caused by serialization errors for checkpoint key.

 

one solution is to clean out the older checkpoints when validation fails due to grouper factory.

  was:
TO change the grouper factory of the job, checkpoint validation should be disabled (using the config = ). This ensures the new checkpoints can be written with the new factory and job can use the new checkpoints. However, since the factory is part of the checkpoint key, the older checkpoints are never cleaned up. This leads to job reading those old checkpoints on every deploy and failing checkpoint validation. to get job running, checkpoint validation needs to be disabled forever.

 

one solution is to clean out the older checkpoints when validation fails due to grouper factory.


> Checkpoint validation enabled needs to be false forever if grouper factory was changed.
> ---------------------------------------------------------------------------------------
>
>                 Key: SAMZA-2664
>                 URL: https://issues.apache.org/jira/browse/SAMZA-2664
>             Project: Samza
>          Issue Type: Bug
>            Reporter: Lakshmi Manasa Gaduputi
>            Assignee: Lakshmi Manasa Gaduputi
>            Priority: Major
>
> TO change the grouper factory of the job, checkpoint validation should be disabled (using the config = ). This ensures the new checkpoints can be written with the new factory and job can use the new checkpoints. However, since the factory is part of the checkpoint key, the older checkpoints are never cleaned up. This leads to job reading those old checkpoints on every deploy and failing checkpoint validation. to get job running, checkpoint validation needs to be disabled forever.
> Keeping checkpoint validation disabled forever is risky, as that would ignore the exceptions caused by serialization errors for checkpoint key.
>  
> one solution is to clean out the older checkpoints when validation fails due to grouper factory.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)