[jira] [Created] (OPTIQ-429) Add statistics SPI for lattice optimization algorithm

["Julian Hyde (JIRA)" <ji...@apache.org>]

Julian Hyde created OPTIQ-429:
---------------------------------

             Summary: Add statistics SPI for lattice optimization algorithm
                 Key: OPTIQ-429
                 URL: https://issues.apache.org/jira/browse/OPTIQ-429
             Project: Optiq
          Issue Type: New Feature
            Reporter: Julian Hyde
            Assignee: Julian Hyde


In OPTIQ-427 we added an an optimization algorithm to choose an initial set of tiles to materialize. The {{rowCountEstimate}} attribute is the number of rows in the lattice.

Add an SPI to generate estimates of (a) the number of rows in the lattice, (b) the number of rows in a given tile of the lattice (specified by its dimensions).

Also add a default implementation of the SPI that executes SQL queries, caches the results, and uses some kind of approximation for the cardinalities of sets of attributes e.g. if quarter has 4 distinct values and year has 10 distinct values then (year, quarter) has 40 distinct values (or perhaps an expectation of 39.79 distinct values in a table with 3,650 rows, per the formula {{n . (1 - ((n - 1) / n) ^ p)}}).

Implementations that read stats from external stats tables, or execute SQL that samples a small percentage of the rows, would also be possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)