You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2022/09/28 14:29:00 UTC
[jira] [Commented] (IMPALA-11418) Optimize select contant statement min memory reservation
[ https://issues.apache.org/jira/browse/IMPALA-11418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17610599#comment-17610599 ]
ASF subversion and git services commented on IMPALA-11418:
----------------------------------------------------------
Commit d47d305bf42f963ba6fac18c014e96c734e9e9ff in impala's branch refs/heads/master from xqhe
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d47d305bf ]
IMPALA-11418: A statement that returns at most one row need not to spool results
A query that returns at most one row can run more efficiently without
result spooling. If result spooling is enabled, it will set the
minimum memory reservation in PlanRootSink, e.g. 'select 1' minimum
memory reservation is 4MB.
This optimization can reduce the statement's resource reservation and
prevent the exception 'Failed to get minimum memory reservation' when
the host memory limit not available.
Testing:
- Add tests in result-spooling.test
Change-Id: Icd4d73c21106048df68a270cf03d4abd56bd3aac
Reviewed-on: http://gerrit.cloudera.org:8080/18711
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Optimize select contant statement min memory reservation
> --------------------------------------------------------
>
> Key: IMPALA-11418
> URL: https://issues.apache.org/jira/browse/IMPALA-11418
> Project: IMPALA
> Issue Type: Improvement
> Components: Frontend
> Affects Versions: Impala 4.0.0, Impala 4.1.0
> Reporter: Xianqing He
> Assignee: Xianqing He
> Priority: Major
> Fix For: Impala 4.2.0
>
>
> If SPOOL_QUERY_RESULTS is true, then the ResourceProfile sets a min reservation in PlanRootSink.
> For the statement 'select 1', the min reservation is 4MB. It's no problem if the cluster has enough mem available within its process limit to execute the query. But if the host mem not available it will throw 'Failed to get minimum memory reservation'.
> Some connection pools use 'select 1' to check whether the connection is available. The check will fail if memory is oversubscribed.
> For this case we can set the min reservation to 0 to reduce failure when the memory is oversubscribed.
>
> {code:java}
> Query: explain select 1
> +--------------------------------------------------------+
> | Explain String |
> +--------------------------------------------------------+
> | Max Per-Host Resource Reservation: Memory=4MB Threads=1 |
> | Per-Host Resource Estimates: Memory=10MB |
> | Codegen disabled by planner |
> | |
> | PLAN-ROOT SINK |
> | | |
> | 00:UNION |
> | constant-operands=1 |
> | row-size=1B cardinality=1 |
> +--------------------------------------------------------+ {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org