You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andrew Or (JIRA)" <ji...@apache.org> on 2015/06/24 03:33:44 UTC

[jira] [Created] (SPARK-8584) Better exception message if invalid checkpoint dir is specified

Andrew Or created SPARK-8584:
--------------------------------

             Summary: Better exception message if invalid checkpoint dir is specified
                 Key: SPARK-8584
                 URL: https://issues.apache.org/jira/browse/SPARK-8584
             Project: Spark
          Issue Type: Bug
          Components: Spark Core
    Affects Versions: 1.0.0
            Reporter: Andrew Or
            Assignee: Andrew Or


If we're running Spark on a cluster, the checkpoint dir must be a non-local path. Otherwise, the attempt to read from a checkpoint will fail because the checkpoint files are written on the executors, not on the driver.

Currently, the error message that you get looks something like the following, which is not super intuitive:
{code}
Checkpoint RDD 3 (0) has different number of partitions than original RDD 2 (100)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org