You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by GitBox <gi...@apache.org> on 2022/10/19 13:15:26 UTC

[GitHub] [flink] kristoffSC opened a new pull request, #21115: [Flink 29627][streaming] Fix duplicate key exception during recover more than 1 committable.

kristoffSC opened a new pull request, #21115:
URL: https://github.com/apache/flink/pull/21115

   ## What is the purpose of the change
   Unchanged backport of https://github.com/apache/flink/pull/21101
   
   Fixing [FLINK-29627](https://issues.apache.org/jira/browse/FLINK-29627) where recovery more than one Committable causesed `IllegalStateException` and prevents cluster to start. 
   
   Implemented fix is based on merging committables from same subtaskId during recovery for Sink V2 architecture.
   
   ## Brief change log
   
     - Enhance `CheckpointSimpleVersionedSerializer::deserialize` by calling `SubtaskCommittableManager::merge` for committables for same subtaskId.
   
   ## Verifying this change
   
   - Add new test `Committablecollectorserializertest::testCommittablesForSameSubtaskIdV2SerDe` to verify deserialization of multiple committables for the same subtaskId.
   - Enhanced `CommitterOperatorTest::testStateRestore` to include committables for same subtaskId.
   
   ## Does this pull request potentially affect one of the following parts:
   
     - Dependencies (does it add or upgrade a dependency): (**no**)
     - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (**no**)
     - The serializers: (**yes**)
     - The runtime per-record code paths (performance sensitive): (**no**)
     - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (**no**)
     - The S3 file system connector: (**no**)
   
   ## Documentation
   
     - Does this pull request introduce a new feature? (**no**)
     - If yes, how is the feature documented? (**not applicable**)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] fapaul merged pull request #21115: [Flink 29627][streaming] Fix duplicate key exception during recovery more than 1 committable.

Posted by GitBox <gi...@apache.org>.
fapaul merged PR #21115:
URL: https://github.com/apache/flink/pull/21115


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [flink] flinkbot commented on pull request #21115: [Flink 29627][streaming] Fix duplicate key exception during recover more than 1 committable.

Posted by GitBox <gi...@apache.org>.
flinkbot commented on PR #21115:
URL: https://github.com/apache/flink/pull/21115#issuecomment-1284017618

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "f9f4f8facf7237c6bc838a719a99d7db9a3b6f7b",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f9f4f8facf7237c6bc838a719a99d7db9a3b6f7b",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f9f4f8facf7237c6bc838a719a99d7db9a3b6f7b UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     The @flinkbot bot supports the following commands:
   
    - `@flinkbot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@flink.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org