[jira] [Updated] (SAMZA-675) Check backwards incompatible config changes

["Naveen Somasundaram (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SAMZA-675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Naveen Somasundaram updated SAMZA-675:
--------------------------------------
    Description: 
One of the nice things we get for coordinator stream is, we have the previous configs we launched the job with. We can use this to our advantage for config validation. For e.g., 
1. The number of partitions in the Kafka stream has changed, this would break our change log partition mapping. 
2. Store was opened with TTL, but reopened again without TTL in RocksDB, this can cause a corruption. 

And many more checks like this. We need a place (probably JobCoordinator) to do these validations on, validate(oldConfig from CoordinatorStream - newConfigsUsers have passed)

  was:
One of the nice things we get for coordinator stream is, we have the previous configs we launched the job with. We can use this to our advantage for config validation. For e.g., 
1. The number of partitions in the Kafka stream has changed, this would break our change log partition mapping. 
2. Store was opened with TTL, but reopened again without TTL in RocksDB, this can cause a corruption. 

And many more checks like this. We need a place (probably JobCoordinator) to do these validations.


> Check backwards incompatible config changes 
> --------------------------------------------
>
>                 Key: SAMZA-675
>                 URL: https://issues.apache.org/jira/browse/SAMZA-675
>             Project: Samza
>          Issue Type: Bug
>          Components: container
>    Affects Versions: 0.10.0
>            Reporter: Naveen Somasundaram
>
> One of the nice things we get for coordinator stream is, we have the previous configs we launched the job with. We can use this to our advantage for config validation. For e.g., 
> 1. The number of partitions in the Kafka stream has changed, this would break our change log partition mapping. 
> 2. Store was opened with TTL, but reopened again without TTL in RocksDB, this can cause a corruption. 
> And many more checks like this. We need a place (probably JobCoordinator) to do these validations on, validate(oldConfig from CoordinatorStream - newConfigsUsers have passed)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)