You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@flink.apache.org by Martin Eden <ma...@gmail.com> on 2017/09/08 08:20:47 UTC

Quick checkpointing related question

Hi all,

I have a Flink 1.3.1 job with a source that implements
CheckpointingFunction.

As I understand it, the notifyCheckpointComplete callback is called when
all the downstream operators in the DAG successfully finished their
checkpoints.

Since I am doing some work in this method, I would like to know if the
latency of the execution of this method is reflected in any of the
checkpointing stats of the source operator? If yes which one? End To End
Duration / Checkpoint Duration sync or async / Alignment duration?

Thanks,
M

Re: Quick checkpointing related question

Posted by Martin Eden <ma...@gmail.com>.

Thanks for the prompt reply Stefan!

On Fri, Sep 8, 2017 at 9:25 AM, Stefan Richter <s....@data-artisans.com>
wrote:

> Hi,
>
> the method is only called after the checkpoint completed on the job
> manager. At this point _all_ work for the checkpoint is done, so doing work
> in this callback does not add any overhead to the checkpoint.
>
> Best,
> Stefan
>
> > Am 08.09.2017 um 10:20 schrieb Martin Eden <ma...@gmail.com>:
> >
> > Hi all,
> >
> > I have a Flink 1.3.1 job with a source that implements
> CheckpointingFunction.
> >
> > As I understand it, the notifyCheckpointComplete callback is called when
> all the downstream operators in the DAG successfully finished their
> checkpoints.
> >
> > Since I am doing some work in this method, I would like to know if the
> latency of the execution of this method is reflected in any of the
> checkpointing stats of the source operator? If yes which one? End To End
> Duration / Checkpoint Duration sync or async / Alignment duration?
> >
> > Thanks,
> > M
> >
> >
>
>

Re: Quick checkpointing related question

Posted by Stefan Richter <s....@data-artisans.com>.

Hi,

the method is only called after the checkpoint completed on the job manager. At this point _all_ work for the checkpoint is done, so doing work in this callback does not add any overhead to the checkpoint.

Best,
Stefan

> Am 08.09.2017 um 10:20 schrieb Martin Eden <ma...@gmail.com>:
> 
> Hi all,
> 
> I have a Flink 1.3.1 job with a source that implements CheckpointingFunction.
> 
> As I understand it, the notifyCheckpointComplete callback is called when all the downstream operators in the DAG successfully finished their checkpoints.
> 
> Since I am doing some work in this method, I would like to know if the latency of the execution of this method is reflected in any of the checkpointing stats of the source operator? If yes which one? End To End Duration / Checkpoint Duration sync or async / Alignment duration?
> 
> Thanks,
> M
> 
>