[GitHub] igalshilman opened a new pull request #7695: [FLINK-11591][core] Support legacy TypeSerializer snapshot transformations

[GitBox <gi...@apache.org>]

igalshilman opened a new pull request #7695: [FLINK-11591][core] Support legacy TypeSerializer snapshot transformations
URL: https://github.com/apache/flink/pull/7695
 
 
   ## What is the purpose of the change
   (copied from the issue description)
   
   In 1.6 and below, the LockableTypeSerializer incorrectly returns directly the element serializer's snapshot instead of wrapping it within an independent snapshot class:
   
   https://github.com/apache/flink/blob/release-1.6/flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/sharedbuffer/Lockable.java#L188
   
   This results in the fact that the the written state information for this would be `(LockableTypeSerializer, SomeArbitrarySnapshot)`.
   
   The problem occurs when restoring this in Flink 1.7+, since compatibility checks are now performed by providing the new serializer to the snapshot, what would happen is:
   `SomeArbitrarySnapshot.resolveSchemaCompatibility(newLockableTypeSerializer)`, which would not work since the arbitrary snapshot does not recognize the LockableTypeSerializer.
   
   To fix this, we essentially need to preprocess that arbitrary snapshot when restoring from <= 1.6 version snapshots.
   
   ## Brief change log
   
   - This PR adds an interface `LegacySerializerSnapshotTransformer` that is being respect by the `TypeSerializerSerializationUtil` that allows serializers provide custom transformation logic for their associated snapshot classes.
   - Uses that interface by the `LockableTypeSerializer`.
   
   ## Verifying this change
   
   This change is already covered by existing tests, such as `CEPMigrationTest`.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (yes / **no**)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**)
     - The serializers: (**yes** / no / don't know)
     - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / **no** / don't know)
     - The S3 file system connector: (yes / **no** / don't know)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (yes / **no**)
     - If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services