You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Tim Armstrong (JIRA)" <ji...@apache.org> on 2019/01/29 09:57:00 UTC
[jira] [Reopened] (IMPALA-691) Process mem limit does not account
for the JVM's memory usage
[ https://issues.apache.org/jira/browse/IMPALA-691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong reopened IMPALA-691:
----------------------------------
> Process mem limit does not account for the JVM's memory usage
> -------------------------------------------------------------
>
> Key: IMPALA-691
> URL: https://issues.apache.org/jira/browse/IMPALA-691
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 1.2.1, Impala 2.0, Impala 2.1, Impala 2.2, Impala 2.3.0
> Reporter: Skye Wanderman-Milne
> Assignee: Tim Armstrong
> Priority: Major
> Labels: incompatibility, resource-management
>
> The JVM doesn't appear to use malloc, so it's memory usage is not reported by tcmalloc and we do not count it in the process mem limit. I verified this by adding a large allocation in the FE, and noting that the total memory usage (virtual or resident) reported in /memz is not affected, but the virtual and resident memory usage reported by top is.
> This is problematic especially because Impala caches table metadata in the FE (JVM) which can become quite big (few GBs) in extreme cases.
> *Workaround*
> As a workaround, we recommend reducing the process memory limit by 1-2GB to "reserve" memory for the JVM. How much memory you should reserve typically depends on the size of your catalog ( number of tables/partitions/columns/blocks etc.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org