You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2018/08/21 18:14:00 UTC
[jira] [Commented] (IMPALA-7462) HASH JOIN takes lot of memory
[ https://issues.apache.org/jira/browse/IMPALA-7462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16587822#comment-16587822 ]
Tim Armstrong commented on IMPALA-7462:
---------------------------------------
The straight_join hit forces the join order to match the textual order of tables in the FROM clause. It's expected that join order will affect memory consumption. A good join order often results in lower memory consumption but it's not the only thing we optimise for.
What's the ask here? If we think that it's generating a bad plan we need more information like an explain plan and ideally table stats to understand what happened.
> HASH JOIN takes lot of memory
> -----------------------------
>
> Key: IMPALA-7462
> URL: https://issues.apache.org/jira/browse/IMPALA-7462
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 2.12.0, Impala 2.13.0
> Reporter: Pranay Singh
> Priority: Major
>
> While running a complex SQL query the peak memory used up by HASH JOIN node is 270 GB
> Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem
> ------------------------------------------------------------------------------------------------------------
> 12:HASH JOIN 4 0.000ns 0.000ns 0 27.89M 270.38 GB 34.00 MB ====>
> When the same query is run with STRAIGHT_JOIN hint the peak memory consumed is 3.25 GB
> Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem
> ------------------------------------------------------------------------------------------------------------
> 11:HASH JOIN 1 10s171ms 10s171ms 1.55M 262.03K 3.25 GB 1.60 GB ===>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org