You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Roman Khachatryan (Jira)" <ji...@apache.org> on 2021/10/01 08:47:00 UTC
[jira] [Commented] (FLINK-24436)
FsStateChangelogWriter#lastAppendedSequenceNumber return different seq
number with no writes
[ https://issues.apache.org/jira/browse/FLINK-24436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423193#comment-17423193 ]
Roman Khachatryan commented on FLINK-24436:
-------------------------------------------
To clarify, there is no assumption that getLastAppendedTo does NOT increment in that case, so correctness not affected. It could cause one unnecessary materialization when it's triggered without state changes.
> FsStateChangelogWriter#lastAppendedSequenceNumber return different seq number with no writes
> --------------------------------------------------------------------------------------------
>
> Key: FLINK-24436
> URL: https://issues.apache.org/jira/browse/FLINK-24436
> Project: Flink
> Issue Type: Bug
> Reporter: Yuan Mei
> Priority: Major
>
> Test code:
> [https://github.com/apache/flink/pull/16606/commits/3d32e902cee493a984bc052b76dfec984743921f]
> LOG:
> {code:java}
> 2157 [main] INFO org.apache.flink.changelog.fs.FsStateChangelogWriter [] - append to 00000000-0000-0000-0000-000000000000: keyGroup=-1 844 bytes
> 2163 [main] INFO org.apache.flink.changelog.fs.FsStateChangelogWriter [] - append to 00000000-0000-0000-0000-000000000000: keyGroup=8 17 bytes
> 2163 [main] INFO org.apache.flink.state.changelog.ChangelogStateBackendTestUtils [] - direct append, last appended to 0
> 2163 [main] INFO org.apache.flink.changelog.fs.FsStateChangelogWriter [] - append to 00000000-0000-0000-0000-000000000000: keyGroup=3 17 bytes
> 2163 [main] INFO org.apache.flink.state.changelog.ChangelogStateBackendTestUtils [] - direct append, last appended to 1
> 2163 [main] INFO org.apache.flink.state.changelog.ChangelogStateBackendTestUtils [] - direct append, last appended to 2
> 2163 [main] INFO org.apache.flink.state.changelog.ChangelogStateBackendTestUtils [] - direct append, last appended to 2
> 2163 [main] INFO org.apache.flink.changelog.fs.FsStateChangelogWriter [] - append to 00000000-0000-0000-0000-000000000000: keyGroup=3 17 bytes
> 2163 [main] INFO org.apache.flink.state.changelog.ChangelogStateBackendTestUtils [] - direct append, last appended to 2
> 2163 [main] INFO org.apache.flink.changelog.fs.FsStateChangelogWriter [] - append to 00000000-0000-0000-0000-000000000000: keyGroup=5 17 bytes
> 2163 [main] INFO org.apache.flink.state.changelog.ChangelogStateBackendTestUtils [] - direct append, last appended to 3
> 2163 [main] INFO org.apache.flink.state.changelog.ChangelogStateBackendTestUtils [] - direct append, last appended to 4
> 2164 [main] INFO org.apache.flink.state.changelog.ChangelogStateBackendTestUtils [] - direct append, last appended to 4
> {code}
> Problem:
> 1. getLastAppendedTo() does not return the same seq number if no writes.
> 2. Materialization depends on `if (upTo.compareTo(changelogSnapshotState.lastMaterializedTo()) > 0)` to decide whether really perform materialisation. This will cause some undefined behavior if I call getLastAppendedTo() twice
--
This message was sent by Atlassian Jira
(v8.3.4#803005)