You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Roman Khachatryan (Jira)" <ji...@apache.org> on 2021/07/09 21:58:00 UTC
[jira] [Created] (FLINK-23344) State ownership: track and discard
private state (rocks, heap)
Roman Khachatryan created FLINK-23344:
-----------------------------------------
Summary: State ownership: track and discard private state (rocks, heap)
Key: FLINK-23344
URL: https://issues.apache.org/jira/browse/FLINK-23344
Project: Flink
Issue Type: Sub-task
Components: Runtime / State Backends
Reporter: Roman Khachatryan
Fix For: 1.14.0
(materialized and non-materialized, only changelog backend)
TM should own changelog backend state to prevent re-uploading state on checkpoint abortion (or missing confirmation). A simpler solution to only own aborted state is less maintanable in the long run.
For that, on TM state should be tracked and discarded (on subsumption+materialization; on shutdown). Both materialized and non-materialized.
See [state ownership design doc|https://docs.google.com/document/d/1NJJQ30P27BmUvD7oa4FChvkYxMEgjRPTVdO1dHLl_9I/edit?usp=sharing], in particular [Tracking private state|https://docs.google.com/document/d/1NJJQ30P27BmUvD7oa4FChvkYxMEgjRPTVdO1dHLl_9I/edit#heading=h.osle3oyzw0bx].
Rescaling not supported until shared state management implemented.
Externalized checkpoints and savepoints should be supported (or please create a separate ticket).
Retained checkpoints is a separated ticket: FLINK-23251
--
This message was sent by Atlassian Jira
(v8.3.4#803005)