You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Alex Gaudio (JIRA)" <ji...@apache.org> on 2014/07/14 06:29:04 UTC

[jira] [Created] (SPARK-2466) Got two different block manager registrations

Alex Gaudio created SPARK-2466:
----------------------------------

             Summary: Got two different block manager registrations
                 Key: SPARK-2466
                 URL: https://issues.apache.org/jira/browse/SPARK-2466
             Project: Spark
          Issue Type: Bug
          Components: Block Manager, Mesos
    Affects Versions: 1.0.0
         Environment: Mesos 0.19.0
Spark 1.0.0
(Ubuntu 14.04 LTS)
            Reporter: Alex Gaudio


On PySpark and SparkR (haven't tried with Scala Spark) running on our Mesos cluster, we get the following error, which causes spark to fail.

```
ERROR BlockManagerMasterActor: Got two different block manager registrations on 20140627-192758-654448812-5050-31629-42
```

We believe this is because tasks between two different stages may share the same task id if they run within the same second.  As a temporary workaround, we are adding a second of space between executions of lazily evaluated spark code.  This appears to solve the problem.

We don't see this issue running spark in local mode.



--
This message was sent by Atlassian JIRA
(v6.2#6252)