You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2021/04/12 00:49:00 UTC
[jira] [Commented] (IMPALA-10445) The ability to adjust NDV's
precision with query option
[ https://issues.apache.org/jira/browse/IMPALA-10445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17318953#comment-17318953 ]
Quanlong Huang commented on IMPALA-10445:
-----------------------------------------
This makes sense to me. Do you plan to work on this soon? If not, I'm going to remove 4.0 from the target version and leave this for 4.1.
> The ability to adjust NDV's precision with query option
> -------------------------------------------------------
>
> Key: IMPALA-10445
> URL: https://issues.apache.org/jira/browse/IMPALA-10445
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 4.0
> Reporter: Fifteen
> Assignee: Fifteen
> Priority: Minor
>
> Since IMPALA-2658, we can trade memory for more accurate NDV estimation. It is fascinating because tests showing error rate within 0.1% while no tremendous resource usage rise is found( #registers is 1 << 18). Users may have less complaint on computation precision in the future.
> However, the road to apply high precision NDV to production environment is uneven.
> 1) We have to re-write sqls for a large number of historical workloads. Which is time costing and is prone to error.
> 2) Cluster users, aka sql writers, are reluctant to lower their expectations. It would be more convenient to have a way for cluster admins to adjust precision for each Admission Control queue according to cluster's resource usage(rough world).
> Propose:
> Add a new query option DEFAULT_NDV_PRECISION to change the default precision setting for NDV()
> Implementation:
> # Add a query option in FE
> # If the option is set, use the matching NDV(,P) function instead of NDV().
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org