Flink checkpoints, is this OK?

[Nikola Milutinovic <n....@levi9.com>]

Hi all.

I would like to ask if what I am seeing is good or not. We are running Flink as Kubernetes session cluster and have checkpoints enabled. When I inspect a checkpoint, I can see only one file: „_metadata“. As I understand it, that is OK, if the state in question is sufficiently small to fi tinto some limit. If not, additional files would be created. I would exepct they are referenced in the _metadata file.

Now, when I peek into the metadata file I can see a lot of segment slike this one:


aced000570

aced000570/XXX_MIDDLEWARE_TEST.dbo.TestTable:0M{"commit_lsn":"00009292:0000bf80:00d9","change_lsn":"00009292:0000bf80:00d9"}-XXX_MIDDLEWARE_TEST.dbo.TestTable?{"type":"CREATE … ,"comment":null}SourceReaderState?xfile:/var/opt/flink-state/flink-checkpoints/3f6599027678003866f9413bbc0bb705/chk-23/4f068ed4-d186-439d-b056-80a22f0c4653nSourceReaderState OPERATOR_STATE_DISTRIBUTION_MODESPLIT_DISTRIBUTEVALUE_SERIALIZERrorg.apache.flink.api.common.typeutils.base.array.BytePrimitiveArraySerializer$By

tePrimitiveArraySerializerSnapshot?x^Lstream-splitM{"commit_lsn":"00009292:0000bf80:00d9","change_lsn":"00009292:0000bf80:00d9"}{"change_lsn":"7f"}-XXX_MIDDLEWARE_TEST.dbo.TestTable/XXX_MIDDLEWARE_TEST.dbo.TestTable:0

So, in there is a reference to a xfile:/var/opt/flink-state/flink-checkpoints/3f6599027678003866f9413bbc0bb705/chk-23/4f068ed4-d186-439d-b056-80a22f0c4653

Well, there is no such file. Is this a correct checkpoint or is it damaged?

Just for context, this is a CDC job reading MS SQL DB and sending CDC records to Kafka. Our checkpoint paramters are:

execution.checkpointing.num-retained, 3
execution.checkpointing.storage, filesystem
execution.checkpointing.dir, file:///var/opt/flink-state/flink-checkpoints
execution.checkpointing.mode, EXACTLY_ONCE
execution.checkpointing.incremental, true
execution.checkpointing.interval, 300000
execution.checkpointing.timeout, 600000
execution.checkpointing.externalized-checkpoint-retention, RETAIN_ON_CANCELLATION
execution.checkpointing.min-pause, 120000


Nix.