You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Manish Maheshwari (Jira)" <ji...@apache.org> on 2022/04/12 13:09:00 UTC
[jira] [Commented] (IMPALA-11238) Avoid the need for COMPUTE STATS for Iceberg tables
[ https://issues.apache.org/jira/browse/IMPALA-11238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17521136#comment-17521136 ]
Manish Maheshwari commented on IMPALA-11238:
--------------------------------------------
Lets see what can be done for this inexpensively - Assuming that iceberg metadata file has NDV for each column populated, can we apply some hueristics or approximation for this to avoid the expensive compute stats.
!image-2022-04-12-21-07-45-357.png!
> Avoid the need for COMPUTE STATS for Iceberg tables
> ---------------------------------------------------
>
> Key: IMPALA-11238
> URL: https://issues.apache.org/jira/browse/IMPALA-11238
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg
> Attachments: image-2022-04-12-21-07-45-357.png
>
>
> We still need to issue COMPUTE STATS for Iceberg tables to do proper planning.
> The main reason for it that Iceberg metadata lacks NDV information about columns at the table level.
> There are plans in Iceberg to store HyperLogLog arrays for data files, so once we have that we could use that information.
> Until that maybe we could use some heuristics from Iceberg metadata when there is no precise NDV available.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org