[GitHub] [flink] banmoy opened a new pull request #9301: [FLINK-13469][state] Ensure resource used by StateMapSnapshot will be…

[GitBox <gi...@apache.org>]

banmoy opened a new pull request #9301: [FLINK-13469][state] Ensure resource used by StateMapSnapshot will be…
URL: https://github.com/apache/flink/pull/9301
 
 
   ## What is the purpose of the change
   
   Currently, resource used by `StateMapSnapshot` may not be released by `StateTableSnapshot` if snapshot fails. This PR will fix this problem.
   
   ## Brief change log
     
   - For `NestedStateMapSnapshot`, there is no resource to release, so  remove `stateMapSnapshot.release()` from `AbstractStateTableSnapshot#writeStateInKeyGroup()`
   - For `CopyOnWriteStateMapSnapshot`, we hope to release the resource as soon as possible after state in a key group is output successfully, so `CopyOnWriteStateTableSnapshot` overrides the method `writeStateInKeyGroup()`, and call `StateMapSnapshot.release()` after state is output
   - `CopyOnWriteStateTableSnapshot` overrides the method `release()` to ensure that resources used by all state map snapshots can be released if checkpoint is cancelled or exception happens where resource can't be released in `writeStateInKeyGroup()`
   - `CopyOnWriteStateMapSnapshot#release()` maybe called twice both in `release()` and `writeStateInKeyGroup()` of `CopyOnWriteStateTableSnapshot`, so we use a flag `CopyOnWriteStateMapSnapshot#released` to ensure the resource will not be released repeatedly
   
   
   ## Verifying this change
   
   This change added tests and can be verified as follows:
     - Added test `CopyOnWriteStateTableTest#testReleaseSnapshotResource` to validates that resources can be released no matter snapshot is successful or failed
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? not applicable

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services