You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Hong Shen (JIRA)" <ji...@apache.org> on 2015/01/27 03:46:34 UTC

[jira] [Created] (SPARK-5421) SparkSql throw OOM at shuffle

Hong Shen created SPARK-5421:
--------------------------------

             Summary: SparkSql throw OOM at shuffle
                 Key: SPARK-5421
                 URL: https://issues.apache.org/jira/browse/SPARK-5421
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 1.2.0
            Reporter: Hong Shen


ExternalAppendOnlyMap if only for the spark job that aggregator isDefined,  but sparkSQL's shuffledRDD haven't define aggregator, so sparkSQL won't spill at shuffle, it's very easy to throw OOM at shuffle.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org