[jira] [Created] (FLINK-34117) CompactCoordinator for table file sink loses data upon job termination

["Alexander Fedulov (Jira)" <ji...@apache.org>]

Alexander Fedulov created FLINK-34117:
-----------------------------------------

             Summary: CompactCoordinator for table file sink loses data upon job termination
                 Key: FLINK-34117
                 URL: https://issues.apache.org/jira/browse/FLINK-34117
             Project: Flink
          Issue Type: Bug
            Reporter: Alexander Fedulov
            Assignee: Alexander Fedulov


CompactCoordinator accumulates data in currentInputFiles and only rolls them into inputFiles in snapshotState(). At the same time it relies on separately receiving checkpoint indications from the upstream operator via processElement() (EndCheckpoint). If the job terminates, the final EndCheckpoint can arrive before the snapshotState() gets called. This leads to data loss (all events in currentInputFiles get discarded).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)