[jira] [Commented] (SPARK-16990) Define the data structure to hold the statistics for CBO

["Ron Hu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SPARK-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422105#comment-15422105 ] 

Ron Hu commented on SPARK-16990:
--------------------------------

We will follow the agreed design spec and add the following information to Statistics class:

-	totalSize: Total uncompressed output data size from a LogicalPlan node.
-	rowCount: Number of output rows from a LogicalPlan node.
-	fileNumber Number of files (or number of HDFS data blocks). Currently only useful when a LogicalPlan is a relation.
-	basicStats: Basic column statistics, i.e., max, min, number of nulls, number of distinct values, max column length, average column length.
-	Histograms: Histograms of columns, i.e., equi-width histogram (for numeric and string types) and equi-height histogram (only for numeric types).
-	histogramMins: Maintain min values for numeric columns.



> Define the data structure to hold the statistics for CBO
> --------------------------------------------------------
>
>                 Key: SPARK-16990
>                 URL: https://issues.apache.org/jira/browse/SPARK-16990
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Optimizer
>    Affects Versions: 2.1.0
>            Reporter: Ron Hu
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org