You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Lingzhe Sun <li...@hirain.com> on 2023/06/29 11:38:12 UTC

checkpoint file deletion

Hi all,

I performed a stateful structure streaming job, and configured 
spark.cleaner.referenceTracking.cleanCheckpoints to true
spark.cleaner.periodicGC.interval to 1min
in the config. But the checkpoint files are not deleted and the number of them keeps growing. Did I miss something?



Lingzhe Sun  
Hirain Technology