[jira] [Commented] (CALCITE-429) Add statistics SPI for lattice optimization algorithm

["Julian Hyde (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CALCITE-429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577857#comment-14577857 ] 

Julian Hyde commented on CALCITE-429:
-------------------------------------

Work started in https://github.com/julianhyde/incubator-calcite/tree/429-lattice-stats-spi.

> Add statistics SPI for lattice optimization algorithm
> -----------------------------------------------------
>
>                 Key: CALCITE-429
>                 URL: https://issues.apache.org/jira/browse/CALCITE-429
>             Project: Calcite
>          Issue Type: New Feature
>            Reporter: Julian Hyde
>            Assignee: Julian Hyde
>
> In OPTIQ-427 we added an an optimization algorithm to choose an initial set of tiles to materialize. The {{rowCountEstimate}} attribute is the number of rows in the lattice.
> Add an SPI to generate estimates of (a) the number of rows in the lattice, (b) the number of rows in a given tile of the lattice (specified by its dimensions).
> Also add a default implementation of the SPI that executes SQL queries, caches the results, and uses some kind of approximation for the cardinalities of sets of attributes e.g. if quarter has 4 distinct values and year has 10 distinct values then (year, quarter) has 40 distinct values (or perhaps an expectation of 39.79 distinct values in a table with 3,650 rows, per the formula {{n . (1 - ((n - 1) / n) ^ p)}}).
> Implementations that read stats from external stats tables, or execute SQL that samples a small percentage of the rows, would also be possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)