You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Wenchen Fan (JIRA)" <ji...@apache.org> on 2016/11/23 13:10:59 UTC

[jira] [Resolved] (SPARK-18522) Create explicit contract for column stats serialization

     [ https://issues.apache.org/jira/browse/SPARK-18522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan resolved SPARK-18522.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 2.1.0

> Create explicit contract for column stats serialization
> -------------------------------------------------------
>
>                 Key: SPARK-18522
>                 URL: https://issues.apache.org/jira/browse/SPARK-18522
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>            Reporter: Reynold Xin
>            Assignee: Reynold Xin
>             Fix For: 2.1.0
>
>
> The current implementation of column stats uses the base64 encoding of the internal UnsafeRow format to persist statistics (in table properties in Hive metastore). This is an internal format that is not stable across different versions of Spark and should NOT be used for persistence.
> In addition, it would be better if statistics stored in the catalog is human readable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org