[jira] [Commented] (FLINK-11711) Add table and column stats

["Flavio Pompermaier (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-11711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16773850#comment-16773850 ] 

Flavio Pompermaier commented on FLINK-11711:
--------------------------------------------

We've just published a computation of those statistics on Flink 1.6.2 at [https://github.com/okkam-it/flink-descriptive-stats.]

Maybe there you can find something useful that you didn't take into account (such as stdDev, kurtosis, skewness, etc)

> Add table and column stats
> --------------------------
>
>                 Key: FLINK-11711
>                 URL: https://issues.apache.org/jira/browse/FLINK-11711
>             Project: Flink
>          Issue Type: New Feature
>          Components: Table API & SQL
>            Reporter: godfrey he
>            Assignee: godfrey he
>            Priority: Major
>
> We define two structure mode to hold statistics
> 1. TableStats: statistics for table level, contains 2 elements:
> rowCount: Long // the number of row count of table
> colStats: Map[String, ColumnStats] // map each column to its ColumnStats
> 2. ColumnStats: statistics for column level, contains 6 elements:
> ndv: Long // number of distinct values
> nullCount: Long // number of null values
> avgLen: Double // average length of column values
> maxLen: Integer // max length of column values
> max: Any // max value of column values
> min: Any // min value of column values



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)