[jira] [Resolved] (IMPALA-2983) Optimize passthrough preaggregations

["Tim Armstrong (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tim Armstrong resolved IMPALA-2983.
-----------------------------------
    Resolution: Later

> Optimize passthrough preaggregations
> ------------------------------------
>
>                 Key: IMPALA-2983
>                 URL: https://issues.apache.org/jira/browse/IMPALA-2983
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Backend
>    Affects Versions: Impala 2.5.0
>            Reporter: Tim Armstrong
>            Priority: Minor
>              Labels: performance
>
> The initial patch for IMPALA-1305 is fairly conservative and leaves a lot of room for improvement. There were some ideas that were shelved because they could cause perf regressions if not carefully implemented.
> * Tune the threshold values better. This is a little tricky since it depends on the cost of exchange, which depends on the cluster properties.
> * Evict some or all partitions from memory to reduce memory overhead and avoid the cost of hash table lookups. The memory reduction is more useful here since the merge agg's hash table inserts will almost certainly be slower than the preaggs hash table lookups.
> * Periodically evict hash table entries to keep the hash tables below a certain threshold



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org