[jira] [Created] (IMPALA-13251) Planner uses session MEM_LIMIT with Max MEM_LIMIT set on Resource Pool

["Andrey Popkov (Jira)" <ji...@apache.org>]

Andrey Popkov created IMPALA-13251:
--------------------------------------

             Summary: Planner uses session MEM_LIMIT with Max MEM_LIMIT set on Resource Pool
                 Key: IMPALA-13251
                 URL: https://issues.apache.org/jira/browse/IMPALA-13251
             Project: IMPALA
          Issue Type: Bug
          Components: Frontend
    Affects Versions: Impala 4.3.0
            Reporter: Andrey Popkov


impalad version 4.3.0

Query is INSERT SELECT FROM WHERE
with no joins, functions, orders etc
fails with OOM

Query MEM_LIMIT is set to 35Gb
Resource Pool max query MEM_LIMIT range is 20Gb

Cluster has 14 hosts and Admission Control is enabled
so we see in query profile:
Query Options (set by configuration): MEM_LIMIT=37580963840, MT_DOP=8, ..
Query Options (set by configuration and planner): MEM_LIMIT=37580963840, MT_DOP=8, ..
Cluster Memory Admitted: 280.00 GB
Max Per-Host Resource Reservation: Memory=128.00MB Threads=8
Per-Host Resource Estimates: Memory=123.54GB
Query(x:y): memory limit exceeded. Limit=20.00 GB Reservation=192.25 MB ReservationLimit=16.00 GB OtherMemory=19.81 GB Total=20.00 GB Peak=20.00 GB

The same query with MEM_LIMIT=0 and MT_DOP=0 is successful with:
Cluster Memory Admitted 280.00 GB
Max Per-Host Resource Reservation: Memory=16.00MB Threads=2
Per-Host Resource Estimates: Memory=35.02GB

The question is why planner uses 35G but not 20G
And perhaps, it has impact on memory utilization



--
This message was sent by Atlassian Jira
(v8.20.10#820010)