You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@spark.apache.org by szhem <gi...@git.apache.org> on 2018/04/02 07:22:04 UTC

[GitHub] spark issue #19373: [SPARK-22150][CORE] PeriodicCheckpointer fails in case o...

Github user szhem commented on the issue:

    https://github.com/apache/spark/pull/19373
  
    > so the fix might be to change to call checkpoint() to checkpoint(eager: true) - this ensures by the time checkpoint call is returned the checkpointing is completed.
    
    Even if checkpoint is completed, `PeriodicRDDCheckpointer` removes files of the checkpointed and materialized RDDs later on, so it may happen that another RDD depends on the already removed files.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org