You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by banu priya <ba...@gmail.com> on 2024/06/19 04:08:57 UTC

Checkpoints and windows size

Hi All,

I have a flink job with key by, tumbling window(2sec window time &uses
processing time)and aggregator.

How often should I run the check point??I don't need the data to be
retained after 2s.

I want to use incremental check point with rocksdb.


Thanks
Banupriya

Re:Re: Checkpoints and windows size

Posted by Feifan Wang <zo...@163.com>.
Hi banu:


> Not all old sst files are present. Few are removed (i think it is because of compaction).


You are right, rocksdb implement delete a key by insert a entry with null value, the space will be release after compaction.


> Now how can I maintain check points size under control??.


Since rocksdb incremental checkpoint directly uploads the sst file (native format) , this generally leads to space amplification. Another factor that affects the checkpoint size when using rocksdb incremental checkpoint is the data compress of rocksdb. In general, it is difficult to accurately estimate the checkpoint size. 
Savepoint with canonical format can truly reflect the size of the state data, however, making such a savepoint is more expensive because all state data must be reorganized into a canonical format. 
I think rocksdb incremental checkpoint is a better choose, although the checkpoint size cannot be calculated accurately, it is generally positively correlated with the actual state size. When the job runs stably, the checkpoint size will fluctuate within a certain range (due to compaction), which can be easily observed during the launch preparation phase. Maybe someone with more experience can give more valuable advice.


> How can I find idle check point size of my project, I found below link but it is not talking about parallelism.


What do you mean "idle checkpoint size" ?



——————————————

Best regards,

Feifan Wang




在 2024-06-19 17:35:13,"banu priya" <ba...@gmail.com> 写道:

Hi Wang,


Thanks a lot for your reply.


Currently I have 2s window and check point interval as 10s. Minimum pass between check point is 5s. What happens is my check points size is growing gradually. I checked the content inside my rocks db local dir and also the shared checkpoints directory. Inside chk-x, I have _metaspace file which shows list of .sst files referred by that check point.


In that I can see that my very old .sst file is still present. I was expecting it to be cleaned.


Not all old sst files are present. Few are removed (i think it is because of compaction).


Now how can I maintain check points size under control??.


How can I find idle check point size of my project, I found below link but it is not talking about parallelism.


https://www.ververica.com/blog/how-to-size-your-apache-flink-cluster-general-guidelines


Any help would really be appreciated :).


Thanks
Banu


On Wed, 19 Jun, 2024, 9:38 am banu priya, <ba...@gmail.com> wrote:

Hi All, 


I have a flink job with key by, tumbling window(2sec window time &uses processing time)and aggregator.


How often should I run the check point??I don't need the data to be retained after 2s.  


I want to use incremental check point with rocksdb. 




Thanks
Banupriya 

Re: Checkpoints and windows size

Posted by banu priya <ba...@gmail.com>.
Hi Wang,

Thanks a lot for your reply.

Currently I have 2s window and check point interval as 10s. Minimum pass
between check point is 5s. What happens is my check points size is growing
gradually. I checked the content inside my rocks db local dir and also the
shared checkpoints directory. Inside chk-x, I have _metaspace file which
shows list of .sst files referred by that check point.

In that I can see that my very old .sst file is still present. I was
expecting it to be cleaned.

Not all old sst files are present. Few are removed (i think it is because
of compaction).

Now how can I maintain check points size under control??.

How can I find idle check point size of my project, I found below link but
it is not talking about parallelism.

https://www.ververica.com/blog/how-to-size-your-apache-flink-cluster-general-guidelines

Any help would really be appreciated :).

Thanks
Banu

On Wed, 19 Jun, 2024, 9:38 am banu priya, <ba...@gmail.com> wrote:

> Hi All,
>
> I have a flink job with key by, tumbling window(2sec window time &uses
> processing time)and aggregator.
>
> How often should I run the check point??I don't need the data to be
> retained after 2s.
>
> I want to use incremental check point with rocksdb.
>
>
> Thanks
> Banupriya
>

Re:Checkpoints and windows size

Posted by Feifan Wang <zo...@163.com>.

Hi banu,


First of all, it should be noted that the checkpoint interval does not affect the state data live time of the window operator. The life cycle of state data is the same as the life cycle of the tumbling window itself. 
A checkpoint is a consistent snapshot of the job ( include state data and other infomation), what it really affects is failover. The longer the checkpoint interval is, the longer the checkpoint used during failover may be from the current time, and the more data needs to be reprocessed. Too short a checkpoint interval may result in excessive checkpoint overhead, so this is a trade-off.


My personal suggestion is to set the checkpoint interval to 5 minutes when using rocksdb incremental checkpoint. You can also make your own choice based on the impacts mentioned above.



——————————————

Best regards,

Feifan Wang




At 2024-06-19 12:08:57, "banu priya" <ba...@gmail.com> wrote:

Hi All, 


I have a flink job with key by, tumbling window(2sec window time &uses processing time)and aggregator.


How often should I run the check point??I don't need the data to be retained after 2s.  


I want to use incremental check point with rocksdb. 




Thanks
Banupriya