You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by "Jeff Zhang (JIRA)" <ji...@apache.org> on 2016/01/22 01:12:40 UTC
[jira] [Updated] (AMBARI-14767) SparkThriftServer may fail to start
when the default memory requirement exceed
yarn.scheduler.maximum-allocation-mb
[ https://issues.apache.org/jira/browse/AMBARI-14767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jeff Zhang updated AMBARI-14767:
--------------------------------
Summary: SparkThriftServer may fail to start when the default memory requirement exceed yarn.scheduler.maximum-allocation-mb (was: SparkThriftServer may fails to start when the default memory requirement exceed yarn.scheduler.maximum-allocation-mb )
> SparkThriftServer may fail to start when the default memory requirement exceed yarn.scheduler.maximum-allocation-mb
> --------------------------------------------------------------------------------------------------------------------
>
> Key: AMBARI-14767
> URL: https://issues.apache.org/jira/browse/AMBARI-14767
> Project: Ambari
> Issue Type: Improvement
> Reporter: Jeff Zhang
> Assignee: Jeff Zhang
>
> {noformat}
> 16/01/22 00:07:33 ERROR SparkContext: Error initializing SparkContext.
> java.lang.IllegalArgumentException: Required executor memory (1024+384 MB) is above the max threshold (1024 MB) of this cluster! Please check the values of 'yarn.scheduler.maximum-allocation-mb' and/or 'yarn.nodemanager.resource.memory-mb'.
> at org.apache.spark.deploy.yarn.Client.verifyClusterResources(Client.scala:283)
> at org.apache.spark.deploy.yarn.Client.submitApplication(Client.scala:139)
> at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:56)
> at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
> at org.apache.spark.SparkContext.<init>(SparkContext.scala:530)
> at org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:56)
> at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:76)
> at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
> at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
> at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)