[jira] [Resolved] (BEAM-9794) Flink pipeline with RequiresStableInput fails after Short.MAX_VALUE checkpoints.

["Maximilian Michels (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-9794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Maximilian Michels resolved BEAM-9794.
--------------------------------------
    Resolution: Fixed

> Flink pipeline with RequiresStableInput fails after Short.MAX_VALUE checkpoints.
> --------------------------------------------------------------------------------
>
>                 Key: BEAM-9794
>                 URL: https://issues.apache.org/jira/browse/BEAM-9794
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>    Affects Versions: 2.14.0, 2.15.0, 2.16.0, 2.17.0, 2.18.0, 2.19.0, 2.20.0
>            Reporter: David Morávek
>            Assignee: Maximilian Michels
>            Priority: Major
>             Fix For: 2.21.0
>
>          Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> Full original report: https://lists.apache.org/thread.html/rb2ebfad16d85bcf668978b3defd442feda0903c20db29c323497a672%40%3Cuser.beam.apache.org%3E
> The exception comes from: https://github.com/apache/flink/blob/release-1.8.0/flink-runtime/src/main/java/org/apache/flink/runtime/state/OperatorBackendSerializationProxy.java#L68
> In the Flink Runner code, each checkpoint results in a new OperatorState (or KeyedState if the stream is keyed):
> https://github.com/apache/beam/blob/v2.14.0/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunner.java#L91-L103
> https://github.com/apache/beam/blob/v2.14.0/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/wrappers/streaming/stableinput/BufferingDoFnRunner.java#L136-L143



--
This message was sent by Atlassian Jira
(v8.3.4#803005)