You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2018/10/05 00:02:00 UTC
[jira] [Commented] (IMPALA-7096) Ensure no memory limit exceeded
regressions from IMPALA-4835 because of non-reserved memory
[ https://issues.apache.org/jira/browse/IMPALA-7096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639068#comment-16639068 ]
ASF subversion and git services commented on IMPALA-7096:
---------------------------------------------------------
Commit 3fabc2de4771349079bcd9dc8bdcb267f43b2a6b in impala's branch refs/heads/master from [~bikram.sngh91]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=3fabc2d ]
IMPALA-7351: Improve memory estimates for Kudu Scan Nodes
This patch adds memory estimates for kudu scan nodes based on
empirically derived estimates for the scan's memory consumption
that were added in IMPALA-7096.
Testing:
Modified resource requirements planner test.
Change-Id: If9bb52530271b0bff91311a67d222a2e9fac1229
Reviewed-on: http://gerrit.cloudera.org:8080/11440
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Ensure no memory limit exceeded regressions from IMPALA-4835 because of non-reserved memory
> -------------------------------------------------------------------------------------------
>
> Key: IMPALA-7096
> URL: https://issues.apache.org/jira/browse/IMPALA-7096
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.13.0, Impala 3.1.0
> Reporter: Tim Armstrong
> Assignee: Tim Armstrong
> Priority: Blocker
> Labels: resource-management
> Fix For: Impala 3.1.0
>
> Attachments: ScanConsumingMostMemory.txt
>
>
> IMPALA-7078 showed some cases where non-buffer memory could accumulate in the row batch queue and cause memory consumption problems.
> The decision for whether to spin up a scanner thread in IMPALA-4835 implicitly assumes that buffer memory is the bulk of memory consumed by a scan, but there may be cases where that is not true and the previous heuristic would be more conservative about starting a scanner thread.
> We should investigate this further and figure out how to avoid it if there's an issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org