[jira] [Commented] (PHOENIX-4164) APPROX_COUNT_DISTINCT becomes imprecise at 20m unique values.

["Lars Hofhansl (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/PHOENIX-4164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154817#comment-16154817 ] 

Lars Hofhansl commented on PHOENIX-4164:
----------------------------------------

Since the correctness drops of steeply, I'm guessing we're running into precision limits of the hash values, and it might simply be a task of increasing that.

I think we have three options:
# Increase the default precision. And _state_ how many unique value we expect to be able to could with that.
# Make the precision configurable (for the record I do not think that's a good idea)
# Leave the precision as is, but still state how unique value this should support.



> APPROX_COUNT_DISTINCT becomes imprecise at 20m unique values.
> -------------------------------------------------------------
>
>                 Key: PHOENIX-4164
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-4164
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>
> {code}
> 0: jdbc:phoenix:localhost> select count(*) from test;
> +-----------+
> | COUNT(1)  |
> +-----------+
> | 26931816  |
> +-----------+
> 1 row selected (14.604 seconds)
> 0: jdbc:phoenix:localhost> select approx_count_distinct(v1) from test;
> +----------------------------+
> | APPROX_COUNT_DISTINCT(V1)  |
> +----------------------------+
> | 17221394                   |
> +----------------------------+
> 1 row selected (21.619 seconds)
> {code}
> The table is generated from random numbers, and the cardinality of v1 is close to the number of rows.
> (I cannot run a COUNT(DISTINCT(v1)), as it uses up all memory on my machine and eventually kills the regionserver - that's another story and another jira)
> [~aertoria]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)