You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@impala.apache.org by jo...@apache.org on 2023/03/22 16:19:51 UTC
[impala] 02/03: IMPALA-11987: [DOCS] Document the mem_limit_executors query option
This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
commit 19b5d781f46798d0ad4fdb4bc8deb352a6d15d07
Author: Shajini Thayasingh <st...@cloudera.com>
AuthorDate: Wed Mar 8 11:20:23 2023 -0800
IMPALA-11987: [DOCS] Document the mem_limit_executors query option
Document the difference between mem_limit and mem_limit_executors
query options.
Change-Id: Ibce8347d564afde0f9b9e83df560454f4b8b5c9d
Reviewed-on: http://gerrit.cloudera.org:8080/19604
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Daniel Becker <da...@cloudera.com>
Reviewed-by: Wenzhe Zhou <wz...@cloudera.com>
---
docs/topics/impala_mem_limit.xml | 21 ++++++++++++++++++++-
1 file changed, 20 insertions(+), 1 deletion(-)
diff --git a/docs/topics/impala_mem_limit.xml b/docs/topics/impala_mem_limit.xml
index aec76e65a..02a1d7750 100644
--- a/docs/topics/impala_mem_limit.xml
+++ b/docs/topics/impala_mem_limit.xml
@@ -209,5 +209,24 @@ for buffer reservations. Memory reservation needed given the current plan: 38.00
or the pool config (max-query-mem-limit, min-query-mem-limit) for the query to allow the query memory limit to be
at least 70.00 MB. Note that changing the mem_limit may also change the plan. See the query profile for more
information about the per-node memory requirements.</codeblock>
- </conbody>
+</conbody>
+<concept id="mem_limit_executors">
+ <title>MEM_LIMIT_EXECUTORS Query Option</title>
+ <conbody>
+ <note>This is a developer-only query option. Setting this query option is not recommended
+ unless specifically advised.</note>
+ <p>The existing <codeph>MEM_LIMIT</codeph> query option applies to all impala coordinators and
+ executors. This means that the same amount of memory gets reserved but coordinators
+ typically just do the job of coordinating the query and thus do not necessarily need all the
+ estimated memory. Blocking the estimated memory on coordinators blocks the memory to be used
+ for other queries.</p>
+ <p>The new <codeph>MEM_LIMIT_EXECUTORS</codeph> query option functions similarly to the
+ <codeph>MEM_LIMIT</codeph> option but sets the query memory limit only on executors. This
+ new option addresses the issue related to <codeph>MEM_LIMIT</codeph> and is recommended in
+ scenarios where the query needs much higher memory on executors compared with
+ coordinators.</p>
+ <p>Note that the <codeph>MEM_LIMIT_EXECUTORS</codeph> option does not work with
+ <codeph>MEM_LIMIT</codeph>. If you set both, only <codeph>MEM_LIMIT</codeph> applies.</p>
+ </conbody>
+</concept>
</concept>