You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Usamah Jassat (Jira)" <ji...@apache.org> on 2023/06/20 10:22:00 UTC

[jira] [Created] (FLINK-32394) Init.pos is partially stored in connector state

Usamah Jassat created FLINK-32394:
-------------------------------------

             Summary: Init.pos is partially stored in connector state
                 Key: FLINK-32394
                 URL: https://issues.apache.org/jira/browse/FLINK-32394
             Project: Flink
          Issue Type: Bug
          Components: Connectors / Kinesis
    Affects Versions: 1.17.1
            Reporter: Usamah Jassat


The Init.pos is partially stored in connector state which can lead to inconsistencies further down the line in idle streams. In particularly an issue arises when the init.pos is AT_TIMESTAMP, and the init.pos is later changed to TRIM_HORIZON.

The issue is that AT_TIMESTAMP is stored in the connector state but the timestamp itself isn't stored in the state. If a stream is idle and the init.pos is changed to TRIM_HORIZON then the connector attempts to read from AT_TIMSTAMP (due to it being stored in state) attempting to get the timestamp from the properties however it is no longer there as the init.pos property is not TRIM_HORIZON.

 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)