You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jean-Georges Perrin <jg...@jgp.net> on 2019/06/27 12:54:00 UTC

Checkpointing and accessing the checkpoint data

Hi Sparkians,

Few questions around checkpointing.

1. Checkpointing “dump” file / persisting to disk
Is the file encrypted or is it a standard parquet file? 

2. If the file is not encrypted, can I use it with another app (I know it’s kind of of a weird stretch case)

3. Have you/do you know of any performance comparison between the two? On small datasets, caching seems more performant, but I can imagine that there is a sweet spot…

Thanks!

jgp



Jean -Georges Perrin
jgp@jgp.net