You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Andrew Or (JIRA)" <ji...@apache.org> on 2015/06/24 03:33:44 UTC
[jira] [Created] (SPARK-8584) Better exception message if invalid
checkpoint dir is specified
Andrew Or created SPARK-8584:
--------------------------------
Summary: Better exception message if invalid checkpoint dir is specified
Key: SPARK-8584
URL: https://issues.apache.org/jira/browse/SPARK-8584
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 1.0.0
Reporter: Andrew Or
Assignee: Andrew Or
If we're running Spark on a cluster, the checkpoint dir must be a non-local path. Otherwise, the attempt to read from a checkpoint will fail because the checkpoint files are written on the executors, not on the driver.
Currently, the error message that you get looks something like the following, which is not super intuitive:
{code}
Checkpoint RDD 3 (0) has different number of partitions than original RDD 2 (100)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org