You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Anton Kalashnikov (Jira)" <ji...@apache.org> on 2021/09/10 09:00:00 UTC
[jira] [Commented] (FLINK-23975) High checkpoint time for unaligned
checkpoint when throthling in source
[ https://issues.apache.org/jira/browse/FLINK-23975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17413059#comment-17413059 ]
Anton Kalashnikov commented on FLINK-23975:
-------------------------------------------
I suppose I have mistaken this scenario. I think that I observed this behavior when StateBackend was set to FileSystemStateBackend and there was not enough memory on the cluster. So it is pretty expected that it would be such degradation for the checkpoint. And of course, it is illegal to compare this number to this fully in-memory working cluster.
I also tried to reproduce that on a similar cluster with the same job and I didn't find any problem. Everything works perfectly fine. So I that we can close this ticket as not reproduced.
> High checkpoint time for unaligned checkpoint when throthling in source
> -----------------------------------------------------------------------
>
> Key: FLINK-23975
> URL: https://issues.apache.org/jira/browse/FLINK-23975
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing, Runtime / Network
> Affects Versions: 1.14.0, 1.13.2
> Reporter: Anton Kalashnikov
> Priority: Major
>
> According to a brief test, if we add a delay in the source before emitting(outside of the checkpoint lock synchronization) it leads to increasing unaligned checkpoint time(in my case from 1sec to 15sec).
> Approximate logic for throttling in source:
>
>
> {noformat}
> @Override
> public void run(SourceContext<Type> sourceContext) throws Exception {
> while (running) {
> LockSupport.parkNanos(sleepNanos);
> synchronized (sourceContext.getCheckpointLock()) {
> sourceContext.collect(new Type(ThreadLocalRandom.current().nextLong(), time, payload));
> }
> }
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)