You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Jimmy Xiang (JIRA)" <ji...@apache.org> on 2014/12/08 18:45:12 UTC
[jira] [Assigned] (HIVE-9040) Spark Memory can be formatted string
[ https://issues.apache.org/jira/browse/HIVE-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jimmy Xiang reassigned HIVE-9040:
---------------------------------
Assignee: Jimmy Xiang
> Spark Memory can be formatted string
> ------------------------------------
>
> Key: HIVE-9040
> URL: https://issues.apache.org/jira/browse/HIVE-9040
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Brock Noland
> Assignee: Jimmy Xiang
>
> Here: https://github.com/apache/hive/blob/spark/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java#L72
> we call {{getInt}} on {{spark.executor.memory}} but this is a formatted string, example here: http://spark.apache.org/docs/1.0.1/configuration.html as such, I get:
> {noformat}
> 2014-12-08 03:04:48,114 WARN [HiveServer2-Handler-Pool: Thread-34]: spark.SetSparkReducerParallelism (SetSparkReducerParallelism.java:process(141)) - Failed to create spark client.
> java.lang.NumberFormatException: For input string: "23000m"
> at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Integer.parseInt(Integer.java:492)
> at java.lang.Integer.parseInt(Integer.java:527)
> at scala.collection.immutable.StringLike$class.toInt(StringLike.scala:229)
> at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
> at org.apache.spark.SparkConf$$anonfun$getInt$2.apply(SparkConf.scala:184)
> at org.apache.spark.SparkConf$$anonfun$getInt$2.apply(SparkConf.scala:184)
> at scala.Option.map(Option.scala:145)
> at org.apache.spark.SparkConf.getInt(SparkConf.scala:184)
> at org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionImpl.getMemoryAndCores(SparkSessionImpl.java:72)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)