You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yun Tang (JIRA)" <ji...@apache.org> on 2019/07/23 17:33:00 UTC
[jira] [Commented] (FLINK-13390) Clarify the exact meaning of state
size when executing incremental checkpoint
[ https://issues.apache.org/jira/browse/FLINK-13390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16891232#comment-16891232 ]
Yun Tang commented on FLINK-13390:
----------------------------------
Another question is whether we should introduce a new rest API or checkpoint stats to track the full state size. Several months ago, for some internal purpose (monitor the overall state size as one of the factors to decide whether to scale up/down), we add a new method named #getFullStateSize() in AbstractCheckpointStats to return the overall state size in our internal Flink. I have discussed with [~srichter] about this before. But from my point of view, not very sure whether this should be contributed back to Flink community as this would only make a difference for RocksDB incremental checkpoint now.
Fundamentally, this problem happened when we introduced PlaceholderStreamStateHandle with zero state size, I even wonder was the meaning of state size changed just an unexpected side-effect?
> Clarify the exact meaning of state size when executing incremental checkpoint
> -----------------------------------------------------------------------------
>
> Key: FLINK-13390
> URL: https://issues.apache.org/jira/browse/FLINK-13390
> Project: Flink
> Issue Type: Improvement
> Reporter: Yun Tang
> Priority: Major
> Fix For: 1.10.0
>
>
> This issue is inspired from [a user mail|https://lists.apache.org/thread.html/56069ce869afbfca66179e89788c05d3b092e3fe363f3540dcdeb7a1@%3Cuser.flink.apache.org%3E] which confused about the state size meaning.
> I think changing the description of state size and add some notices on documentation could help this. Moreover, change the log when complete checkpoint should be also taken into account.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)