You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by Dan Hill <qu...@gmail.com> on 2021/02/18 06:35:03 UTC
Best practices around checkpoint intervals and sizes?
Hi. I'm playing around with optimizing our checkpoint intervals and sizes.
Are there any best practices around this? I have a ~7 sequential joins and
a few sinks. I'm curious what would result in the better throughput and
latency trade offs. I'd assume less frequent checkpointing would increase
throughput (but constrained by how frequently I want checkedpointed sinks
written).
Re: Best practices around checkpoint intervals and sizes?
Posted by Chesnay Schepler <ch...@apache.org>.
A lower checkpoint interval (== more checkpoints / time) will consume
more resources and hence can affect the job performance.
It ultimately boils down to how much latency you are willing to accept
when a failure occurs and data has to be re-processed (more checkpoints
=> less data).
How long this catch-up takes depends on the job and provisioning of the
cluster. An over-provisioned cluster can recover quicker from what is
ultimately just a data spike, while one that is barely keeping up may
incur significant latency.
We know that many users have a checkpointing interval of the order of
minutes, but at the end of the day you will need to run some experiments
with your job&cluster&data to get some rough numbers.
On 2/18/2021 7:35 AM, Dan Hill wrote:
> Hi. I'm playing around with optimizing our checkpoint intervals and
> sizes.
>
> Are there any best practices around this? I have a ~7 sequential
> joins and a few sinks. I'm curious what would result in the better
> throughput and latency trade offs. I'd assume less frequent
> checkpointing would increase throughput (but constrained by how
> frequently I want checkedpointed sinks written).