You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Liang-Chi Hsieh (JIRA)" <ji...@apache.org> on 2016/04/22 09:10:13 UTC

[jira] [Updated] (SPARK-14838) Implement statistics in SerializeFromObject to avoid failure when estimating sizeInBytes for ObjectType

     [ https://issues.apache.org/jira/browse/SPARK-14838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Liang-Chi Hsieh updated SPARK-14838:
------------------------------------
    Summary: Implement statistics in SerializeFromObject to avoid failure when estimating sizeInBytes for ObjectType  (was: Skip automatically broadcast a plan when it contains ObjectProducer)

> Implement statistics in SerializeFromObject to avoid failure when estimating sizeInBytes for ObjectType
> -------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-14838
>                 URL: https://issues.apache.org/jira/browse/SPARK-14838
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>            Reporter: Liang-Chi Hsieh
>
> Spark will determine the plan size to automatically broadcast it or not when doing join. As it can't estimate object type size, this mechanism will throw failure as shown in https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56533/consoleFull. We should fix it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org