You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Davies Liu (JIRA)" <ji...@apache.org> on 2015/11/18 04:51:11 UTC
[jira] [Resolved] (SPARK-11737) String may not be serialized
correctly with Kyro
[ https://issues.apache.org/jira/browse/SPARK-11737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Davies Liu resolved SPARK-11737.
--------------------------------
Resolution: Fixed
Fix Version/s: 1.5.2
1.6.0
Issue resolved by pull request 9704
[https://github.com/apache/spark/pull/9704]
> String may not be serialized correctly with Kyro
> ------------------------------------------------
>
> Key: SPARK-11737
> URL: https://issues.apache.org/jira/browse/SPARK-11737
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.1, 1.6.0
> Reporter: Davies Liu
> Assignee: Davies Liu
> Priority: Critical
> Fix For: 1.6.0, 1.5.2
>
>
> When run in cluster mode, the driver may have different memory (and configs) than executor, also if Kyro is used, then string can not be collected back to driver:
> {code}
> >>> sqlContext.range(10).selectExpr("repeat(cast(id as string), 9)").show()
> +----------------------------+
> |repeat(cast(id as string),9)|
> +----------------------------+
> | 0|
> | 1|
> | 2|
> | 3|
> | 4|
> | 5|
> | 6|
> | 7|
> | 8|
> | 9|
> +----------------------------+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org