You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/09/07 21:21:01 UTC
[jira] [Commented] (IMPALA-7547) For distinct queries use
dictionary encoded page instead of reading all data
[ https://issues.apache.org/jira/browse/IMPALA-7547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607675#comment-16607675 ]
Tim Armstrong commented on IMPALA-7547:
---------------------------------------
I think this is kind of similar to IMPALA-4986 - it requires a similar transformation. Would have to be careful parsing the Parquet spec - it might technically allow values in the dictionary that aren't present in the data.
> For distinct queries use dictionary encoded page instead of reading all data
> ----------------------------------------------------------------------------
>
> Key: IMPALA-7547
> URL: https://issues.apache.org/jira/browse/IMPALA-7547
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Peter Ebert
> Priority: Major
>
> When dictionary encoding is in use the lookup table should contain a distinct list of all values in the data, can skip reading the values and just read the header to get distinct values.
>
> Realize this would be a big change to the read/scanner threads but could greatly speed up distinct queries.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org