You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2019/02/27 13:11:39 UTC

[GitHub] tzulitai opened a new pull request #7849: [FLINK-11772] [DataStream] InternalTimerServiceSerializationProxy should not be using the new serialization compatibility abstractions

tzulitai opened a new pull request #7849: [FLINK-11772] [DataStream] InternalTimerServiceSerializationProxy should not be using the new serialization compatibility abstractions
URL: https://github.com/apache/flink/pull/7849
 
 
   ## What is the purpose of the change
   
   This PR is based on #7818. The first commit is not relevant.
   
   All of the changes done to managed state surrounding:
   - how we no longer Java-serialize serializers anymore, and only write the serializer
   snapshot
   - All schema compatibility checks of serializers go through `TypeSerializerSnapshot#resolveSchemaCompatibility(TypeSerializer)`
   were not reflected to the timers, due to the fact that timers were not handled by state backends (and were therefore not managed state) in the past, and were handled in an
   isolated manner by the {{InternalTimerServiceSerializationProxy}}.
   
   This PR updates the {{InternalTimerServiceSerializationProxy}} accordingly.
   
   ## Brief change log
   
   - ad7222e: Minor code cleanup related to method / field names
   - 7bc827b: This commit fixes compatibility checks of key / namespace serializers to go through `TypeSerializerSnapshot#resolveSchemaCompatibility`, instead of the broken `CompatibilityUtil`. It also handles serializer reconfiguration properly.
   - d0b5080: This commit upticks the version of {{InternalTimerServiceSerializationProxy}} to 2, and also introduces new `InternalTimersSnapshotWriter` and `InternalTimersSnapshotReader` for version 2, that correctly only writes serializer snapshots as state.
   
   ## Verifying this change
   
   Existing migration IT cases that use timers, such as `WindowOperatorMigrationTest` and `CEPMigrationTest` should cover this change.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): no
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: no
     - The serializers: no
     - The runtime per-record code paths (performance sensitive): no
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: yes
     - The S3 file system connector: no
   
   ## Documentation
   
     - Does this pull request introduce a new feature? no
     - If yes, how is the feature documented? N/A
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services