You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Bikramjeet Vig (JIRA)" <ji...@apache.org> on 2018/08/08 17:36:00 UTC
[jira] [Commented] (IMPALA-7361) test_heterogeneous_proc_mem_limit:
Query aborted: Not enough memory available on host (s3)
[ https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16573557#comment-16573557 ]
Bikramjeet Vig commented on IMPALA-7361:
----------------------------------------
I looked at the logs and it seems like the instance of the first query that ran in this test (which reserves 2G of memory on each host) never completed on the host on the port 22001. Since admission controller looks at the max of (mem_admitted[host], mem_reserved[host]) == max (0, 2GB) it queued the query based on that and it eventually timed out with the wrong error message.
Note: The mem reserved for a query that is currently executing is its limit_, if set (which should be the common case with admission control). Otherwise, if the query has no limit or the query is finished executing, the current consumption is used.
> test_heterogeneous_proc_mem_limit: Query aborted: Not enough memory available on host (s3)
> -------------------------------------------------------------------------------------------
>
> Key: IMPALA-7361
> URL: https://issues.apache.org/jira/browse/IMPALA-7361
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.1.0
> Reporter: nithya
> Assignee: Bikramjeet Vig
> Priority: Major
> Labels: broken-build
>
> texttest_heterogeneous_proc_mem_limit custom cluster test fails with the following assertion error
> Stacktrace
> {noformat}
> custom_cluster/test_admission_controller.py:514: in test_heterogeneous_proc_mem_limit
> assert re.search("Queued reason: Not enough memory available on host \S+.Needed "
> E AssertionError: ImpalaBeeswaxException:
> E Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reason: Not enough memory available on host impala-ec2-centos74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.
> E
> E
> E assert None
> E + where None = <function search at 0x7f2b4a67c5f0>('Queued reason: Not enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.\n\n')
> E + where <function search at 0x7f2b4a67c5f0> = re.search
> E + and 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reaso...os74-m5-4xlarge-ondemand-08d6.vpc.cloudera.com:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org