You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Sahil Takiar (Jira)" <ji...@apache.org> on 2020/05/15 19:04:00 UTC
[jira] [Created] (IMPALA-9754) buffer_pool_limit error message is
confusing
Sahil Takiar created IMPALA-9754:
------------------------------------
Summary: buffer_pool_limit error message is confusing
Key: IMPALA-9754
URL: https://issues.apache.org/jira/browse/IMPALA-9754
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Sahil Takiar
The error message for an invalid value of {{buffer_pool_limit}} is confusing. For example:
{code:java}
I0515 18:30:20.143811 1 exec-env.cc:480] System memory available: 30.63 GB (from physical mem)
I0515 18:30:20.144037 1 exec-env.cc:487] CGroup memory limit for this process reduces physical memory available to: 9.31 GB
I0515 18:30:20.144050 1 exec-env.cc:503] Using process memory limit: 7.45 GB (--mem_limit=80% of 9.31 GB)
I0515 18:30:20.150154 1 status.cc:129] Invalid --buffer_pool_limit value, must be a percentage or positive bytes value or percentage: 85%
@ 0xc33b79
@ 0x111afb8
@ 0x1251256
@ 0xb730b3
@ 0x7f60da4c9554
@ 0xbf3e86
F0515 18:30:20.150177 1 impalad-main.cc:73] Invalid --buffer_pool_limit value, must be a percentage or positive bytes value or percentage: 85%
. Impalad exiting. {code}
1: "must be a percentage or positive bytes value or percentage" needs to be revised to something like "must be a positive bytes value or percentage"
2: The error message makes it sound like the value of 85% is invalid, when in reality, it is a valid value - Impala does some internal calculations of what the actual {{buffer_pool_limit}} should be internally (see ExecEnv::Init) - these calculations first compute the mem limit for the process (ChooseProcessMemLimit) based on the physical machines memory + cgroup settings, and then if mem_limit_includes_jvm == true it subtracts the value of Xmx from the amount of available memory - we don't have logic to capture if the results of that subtraction becomes a negative number, which causes the buffer_pool_limit parsing to fail in a weird way. We should add a check to see if {{admit_mem_limit_ \-= JvmMemoryMetric::HEAP_MAX_USAGE->GetValue()}} is negative, and then exit with an informative status message
--
This message was sent by Atlassian Jira
(v8.3.4#803005)