You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2026/03/23 15:31:51 UTC

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24129


Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................

IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Impala should use memory-based query admission when Max Memory is set
for the given pool. However, if Min Query Memory Limit, Max Query Memory
Limit are missing, and query option MEM_LIMIT is not set, then currently
queries can run with unlimited resources.

This can cause a situation when Max Memory is specified, but the pool
can consume basically unlimited memory.

This patch fixes ScheduleState::UpdateMemoryRequirements() to not turn
off memory-based query admission when Max Memory is set.

The patch also removes an unused parameter of
AdmissionController::CanAdmitRequest().

Testing
* new admission controller backend tests added

Change-Id: I4834964e4361895e10627a661831253ce676c129
---
M be/src/scheduling/admission-controller-test.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/schedule-state.cc
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
5 files changed, 134 insertions(+), 30 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/24129/1
-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 3:

(6 comments)

Sorry for the late reply, thanks for the comments!

http://gerrit.cloudera.org:8080/#/c/24129/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/24129/2//COMMIT_MSG@7
PS2, Line 7: IMPALA-14840: Enforce pool Max Memory when query limits are unset
> nit: The commit message subject is a bit long (98 characters). 
We only enforce the 72-char limit on the body, title can be longer. But I agree that the proposed title is better.


http://gerrit.cloudera.org:8080/#/c/24129/2//COMMIT_MSG@20
PS2, Line 20: The patch also removes the unused 'root_cfg' parameter of
            : AdmissionController::CanAdmitRequest().
> nit: Explicitly naming the removed parameter (root_cfg) might be better, so
Done


http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller-test.cc
File be/src/scheduling/admission-controller-test.cc:

http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller-test.cc@2288
PS2, Line 2288: ds), ensur
> nit: The inline comment // > 2 GB / 2  is mathematically correct but a bit 
Done


http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller.h
File be/src/scheduling/admission-controller.h:

http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller.h@193
PS2, Line 193: no m
> small nit: The comment here originally said "(If both these max/min limits 
Reworded a bit.


http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/schedule-state.cc
File be/src/scheduling/schedule-state.cc:

http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/schedule-state.cc@305
PS2, Line 305:   const bool mimic_old_behaviour =
             :       pool_cfg.max_mem_resources <= 0 &&
             :       pool_cfg.min_query_mem_limit == 0 &&
> nit: Maybe aligning the boolean conditions here might be better for visual 
Done


http://gerrit.cloudera.org:8080/#/c/24129/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/24129/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@809
PS2, Line 809: Math.max(numExecutorsForPlanning(), 1)
> Great catch of numExecutorsForPlanning() division-by-zero in planning phase
Thanks! :)



-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 Apr 2026 15:41:24 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/22098/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Thu, 09 Apr 2026 16:02:09 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/21999/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Mon, 23 Mar 2026 15:57:38 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................

IMPALA-14840: Enforce pool Max Memory when query limits are unset

Impala should use memory-based query admission when Max Memory is set
for the given pool. However, if Min Query Memory Limit, Max Query Memory
Limit are missing, and query option MEM_LIMIT is not set, then currently
queries can run with unlimited resources.

This can cause a situation when Max Memory is specified, but the pool
can consume basically unlimited memory.

This patch fixes ScheduleState::UpdateMemoryRequirements() to not turn
off memory-based query admission when Max Memory is set.

The patch also removes the unused 'root_cfg' parameter of
AdmissionController::CanAdmitRequest().

Testing
* new admission controller backend tests added
* updated pool 'poolNoMemLimits' in mem-limit-test-fair-scheduler.xml
  to not specify Max Memory

Change-Id: I4834964e4361895e10627a661831253ce676c129
Reviewed-on: http://gerrit.cloudera.org:8080/24129
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/scheduling/admission-controller-test.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/schedule-state.cc
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/test/resources/mem-limit-test-fair-scheduler.xml
M tests/custom_cluster/test_admission_controller.py
7 files changed, 144 insertions(+), 37 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 5
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Surya Hebbar (Code Review)" <ge...@cloudera.org>.
Surya Hebbar has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 3: Code-Review+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 10 Apr 2026 11:12:30 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 1: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/13267/


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Tue, 24 Mar 2026 14:19:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13315/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Apr 2026 09:06:13 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Yida Wu (Code Review)" <ge...@cloudera.org>.
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 2: Code-Review+1

Looks good to me, have we passed exhaustive tests? I think some admission controller tests are in exhaustive that might be involved


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Wed, 25 Mar 2026 22:28:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Yida Wu (Code Review)" <ge...@cloudera.org>.
Yida Wu has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Apr 2026 22:26:18 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 2:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13269/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Tue, 24 Mar 2026 16:12:45 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 4: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Apr 2026 11:55:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Surya Hebbar (Code Review)" <ge...@cloudera.org>.
Surya Hebbar has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 2: Code-Review+1

(6 comments)

Thank your for the patch. A very clean fix for the resource leakage!

I've left a few minor nits inline mostly regarding comment clarity and commit message formatting.

Overall, the logic looks solid. LGTM!

http://gerrit.cloudera.org:8080/#/c/24129/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/24129/2//COMMIT_MSG@7
PS2, Line 7: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
nit: The commit message subject is a bit long (98 characters). 

Please consider shortening it to fit standard git log formatting.

e.g. IMPALA-14840: Enforce pool Max Memory when query limits are unset


http://gerrit.cloudera.org:8080/#/c/24129/2//COMMIT_MSG@20
PS2, Line 20: The patch also removes an unused parameter of
            : AdmissionController::CanAdmitRequest().
nit: Explicitly naming the removed parameter (root_cfg) might be better, so it's easily searchable in the git history:

"The patch also removes the unused root_cfg parameter of AdmissionController::CanAdmitRequest()."


http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller-test.cc
File be/src/scheduling/admission-controller-test.cc:

http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller-test.cc@2288
PS2, Line 2288: > 2 GB / 2
nit: The inline comment // > 2 GB / 2  is mathematically correct but a bit brief.

Can it be expanded slightly so the reader immediately understands the constraint we are testing against? Maybe something like the following,

// 3 GB is > (pool_max_mem / num_backends), ensuring the estimate exceeds the per-node pool budget


http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller.h
File be/src/scheduling/admission-controller.h:

http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller.h@193
PS2, Line 193: both
small nit: The comment here originally said "(If both these max/min limits are not configured...)". Since we are now checking three properties (min_query_mem_limit, max_query_mem_limit, and max_mem_resources), consider removing the word "both".

...

(If these max/min limits are not configured, and there is no max_mem_resources configured for the pool, then...)


http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/schedule-state.cc
File be/src/scheduling/schedule-state.cc:

http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/schedule-state.cc@305
PS2, Line 305:   const bool mimic_old_behaviour =
             :       pool_cfg.max_mem_resources <= 0 &&
             :       pool_cfg.min_query_mem_limit == 0 && pool_cfg.max_query_mem_limit == 0;
nit: Maybe aligning the boolean conditions here might be better for visual flow, or wrapping them if they exceed the column limit.

const bool mimic_old_behaviour =
    pool_cfg.max_mem_resources <= 0 &&
    pool_cfg.min_query_mem_limit == 0 && 
    pool_cfg.max_query_mem_limit == 0;


http://gerrit.cloudera.org:8080/#/c/24129/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java
File fe/src/main/java/org/apache/impala/analysis/Analyzer.java:

http://gerrit.cloudera.org:8080/#/c/24129/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@809
PS2, Line 809: Math.max(numExecutorsForPlanning(), 1)
Great catch of numExecutorsForPlanning() division-by-zero in planning phase!



-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2026 20:47:44 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/22009/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Tue, 24 Mar 2026 16:20:03 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 2: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Tue, 24 Mar 2026 20:37:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Surya Hebbar (Code Review)" <ge...@cloudera.org>.
Surya Hebbar has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 2:

(2 comments)

Thank you for resolving the comments!

:)

LGTM!

http://gerrit.cloudera.org:8080/#/c/24129/2//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/24129/2//COMMIT_MSG@7
PS2, Line 7: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
> We only enforce the 72-char limit on the body, title can be longer. But I a
Thanks!


http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller.h
File be/src/scheduling/admission-controller.h:

http://gerrit.cloudera.org:8080/#/c/24129/2/be/src/scheduling/admission-controller.h@193
PS2, Line 193: both
> Reworded a bit.
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 10 Apr 2026 11:12:17 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 4: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Apr 2026 07:40:42 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello Yida Wu, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/24129

to look at the new patch set (#2).

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................

IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Impala should use memory-based query admission when Max Memory is set
for the given pool. However, if Min Query Memory Limit, Max Query Memory
Limit are missing, and query option MEM_LIMIT is not set, then currently
queries can run with unlimited resources.

This can cause a situation when Max Memory is specified, but the pool
can consume basically unlimited memory.

This patch fixes ScheduleState::UpdateMemoryRequirements() to not turn
off memory-based query admission when Max Memory is set.

The patch also removes an unused parameter of
AdmissionController::CanAdmitRequest().

Testing
* new admission controller backend tests added
* updated pool 'poolNoMemLimits' in mem-limit-test-fair-scheduler.xml
  to not specify Max Memory

Change-Id: I4834964e4361895e10627a661831253ce676c129
---
M be/src/scheduling/admission-controller-test.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/schedule-state.cc
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/test/resources/mem-limit-test-fair-scheduler.xml
M tests/custom_cluster/test_admission_controller.py
7 files changed, 141 insertions(+), 36 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/24129/2
-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 1:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13267/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Comment-Date: Tue, 24 Mar 2026 09:53:27 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 13 Apr 2026 13:24:07 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Hello Yida Wu, Surya Hebbar, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/24129

to look at the new patch set (#3).

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................

IMPALA-14840: Enforce pool Max Memory when query limits are unset

Impala should use memory-based query admission when Max Memory is set
for the given pool. However, if Min Query Memory Limit, Max Query Memory
Limit are missing, and query option MEM_LIMIT is not set, then currently
queries can run with unlimited resources.

This can cause a situation when Max Memory is specified, but the pool
can consume basically unlimited memory.

This patch fixes ScheduleState::UpdateMemoryRequirements() to not turn
off memory-based query admission when Max Memory is set.

The patch also removes the unused 'root_cfg' parameter of
AdmissionController::CanAdmitRequest().

Testing
* new admission controller backend tests added
* updated pool 'poolNoMemLimits' in mem-limit-test-fair-scheduler.xml
  to not specify Max Memory

Change-Id: I4834964e4361895e10627a661831253ce676c129
---
M be/src/scheduling/admission-controller-test.cc
M be/src/scheduling/admission-controller.cc
M be/src/scheduling/admission-controller.h
M be/src/scheduling/schedule-state.cc
M fe/src/main/java/org/apache/impala/analysis/Analyzer.java
M fe/src/test/resources/mem-limit-test-fair-scheduler.xml
M tests/custom_cluster/test_admission_controller.py
7 files changed, 144 insertions(+), 37 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/29/24129/3
-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>

[Impala-ASF-CR] IMPALA-14840: Enforce pool Max Memory when query limits are unset

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Enforce pool Max Memory when query limits are unset
......................................................................


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/13321/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 4
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Tue, 14 Apr 2026 07:40:43 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set

Posted by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org>.
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/24129 )

Change subject: IMPALA-14840: Pool can use more memory than Max Memory when Min/Max Query Memory limit is not set
......................................................................


Patch Set 2:

> Looks good to me, have we passed exhaustive tests? I think some admission controller tests are in exhaustive that might be involved

Thanks for taking a look! Exhaustive tests passed.


-- 
To view, visit http://gerrit.cloudera.org:8080/24129
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4834964e4361895e10627a661831253ce676c129
Gerrit-Change-Number: 24129
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Surya Hebbar <sh...@cloudera.com>
Gerrit-Reviewer: Yida Wu <wy...@gmail.com>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Fri, 27 Mar 2026 10:05:00 +0000
Gerrit-HasComments: No