You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Jean-Georges Perrin <jg...@jgp.net> on 2019/06/27 12:54:00 UTC
Checkpointing and accessing the checkpoint data
Hi Sparkians,
Few questions around checkpointing.
1. Checkpointing “dump” file / persisting to disk
Is the file encrypted or is it a standard parquet file?
2. If the file is not encrypted, can I use it with another app (I know it’s kind of of a weird stretch case)
3. Have you/do you know of any performance comparison between the two? On small datasets, caching seems more performant, but I can imagine that there is a sweet spot…
Thanks!
jgp
Jean -Georges Perrin
jgp@jgp.net