[jira] [Commented] (SPARK-11130) TestHive fails on machines with few cores

["Andrew Or (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-11130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14999661#comment-14999661 ] 

Andrew Or commented on SPARK-11130:
-----------------------------------

[~vanzin] is this resolved through SPARK-11251 already?

> TestHive fails on machines with few cores
> -----------------------------------------
>
>                 Key: SPARK-11130
>                 URL: https://issues.apache.org/jira/browse/SPARK-11130
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 1.5.0, 1.6.0
>            Reporter: Marcelo Vanzin
>            Priority: Minor
>
> Filing so it doesn't get lost (again).
> TestHive.scala has this code:
> {code}
>     new SparkContext(
>       System.getProperty("spark.sql.test.master", "local[32]"),
> {code}
> On machines with less cores, that causes many tests to fail with "unable to allocate memory" errors, because the default page size calculation seems to be based on the machine's core count, and not on the core count specified for the SparkContext.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org