You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "David Knupp (JIRA)" <ji...@apache.org> on 2018/07/31 21:49:00 UTC
[jira] [Updated] (IMPALA-7361) test_heterogeneous_proc_mem_limit:
Query aborted:
[ https://issues.apache.org/jira/browse/IMPALA-7361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
David Knupp updated IMPALA-7361:
--------------------------------
Summary: test_heterogeneous_proc_mem_limit: Query aborted: (was: test_heterogeneous_proc_mem_limit: Assertion Failure)
> test_heterogeneous_proc_mem_limit: Query aborted:
> ---------------------------------------------------
>
> Key: IMPALA-7361
> URL: https://issues.apache.org/jira/browse/IMPALA-7361
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: nithya
> Assignee: Bikramjeet Vig
> Priority: Major
> Labels: broken-build
>
> test_heterogeneous_proc_mem_limit fails with the following assertion error
>
> {code:java}
> AssertionError: ImpalaBeeswaxException: Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reason: Not enough memory available on host <hostname>:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available. assert None + where None = <function search at 0x7f2b4a67c5f0>('Queued reason: Not enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reaso...<hostname>:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.\n\n') + where <function search at 0x7f2b4a67c5f0> = re.search + and 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reaso...<hostname>:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
> {code}
>
> stack trace
> {code:java}
> *Stacktrace*
> custom_cluster/test_admission_controller.py:514: in test_heterogeneous_proc_mem_limit
> assert re.search("Queued reason: Not enough memory available on host \S+.Needed "
> E AssertionError: ImpalaBeeswaxException:
> E Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reason: Not enough memory available on host <host>:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.
> E
> E
> E assert None
> E + where None = <function search at 0x7f2b4a67c5f0>('Queued reason: Not enough memory available on host \\S+.Needed 2.00 GB but only 1.00 GB out of 2.00 GB was available.', 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reaso...<hostname>:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.\n\n')
> E + where <function search at 0x7f2b4a67c5f0> = re.search
> E + and 'ImpalaBeeswaxException:\n Query aborted:Admission for query exceeded timeout 200ms in pool default-pool. Queued reaso...<hostname>:22001.Needed 2.00 GB but only 1.00 GB out of 3.00 GB was available.\n\n' = str(ImpalaBeeswaxException())
> *Standard Error*
> 08:55:51 MainThread: Starting State Store logging to /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 08:55:52 MainThread: Starting Catalog Service logging to /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 08:55:53 MainThread: Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 08:55:54 MainThread: Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 08:55:55 MainThread: Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-master-core-s3/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 08:55:58 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 08:55:58 MainThread: Getting num_known_live_backends from <hostname>:25000
> 08:55:58 MainThread: Waiting for num_known_live_backends=3. Current value: 0
> 08:55:59 MainThread: Getting num_known_live_backends from<hostname>:25000
> 08:55:59 MainThread: Waiting for num_known_live_backends=3. Current value: 1
> 08:56:00 MainThread: Getting num_known_live_backends from <hostname>:25000
> 08:56:00 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 08:56:01 MainThread: Getting num_known_live_backends from <hostname>:25000
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Getting num_known_live_backends from <hostname>:25001
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Getting num_known_live_backends from <hostname>:25002
> 08:56:01 MainThread: num_known_live_backends has reached value: 3
> 08:56:01 MainThread: Impala Cluster Running with 3 nodes (3 coordinators, 3 executors).
> MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> MainThread: Getting metric: statestore.live-backends from <hostname>:25010
> MainThread: Metric 'statestore.live-backends' has reached desired value: 4
> MainThread: Getting num_known_live_backends from <hostname>:25000
> MainThread: num_known_live_backends has reached value: 3
> MainThread: Getting num_known_live_backends from <hostname>:25001
> MainThread: num_known_live_backends has reached value: 3
> MainThread: Getting num_known_live_backends from <hostname>:25002
> MainThread: num_known_live_backends has reached value: 3
> -- connecting to: localhost:21000
> -- executing against localhost:21000
> use default;
>
> SET sync_ddl=1;
> -- executing against localhost:21000
> drop database if exists `hs2_db` cascade;
>
> SET disable_codegen_rows_threshold=5000;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> SET mem_limit=2G;
> SET batch_size=0;
> SET num_nodes=0;
> -- executing against localhost:21000
> select * from functional.alltypesagg, (select 1) B limit 1;
>
> SET disable_codegen_rows_threshold=5000;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> SET mem_limit=3G;
> SET batch_size=0;
> SET num_nodes=1;
> -- executing against localhost:21000
> select * from functional.alltypesagg, (select 1) B limit 1;
>
> SET disable_codegen_rows_threshold=5000;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> SET mem_limit=3G;
> SET batch_size=0;
> SET num_nodes=0;
> -- executing against localhost:21000
> select * from functional.alltypesagg, (select 1) B limit 1;
>
> -- connecting to: <hostname>:21002
> SET mem_limit=1G;
> -- executing async: <hostname>:21002
> select sleep(1000);
>
> SET disable_codegen_rows_threshold=5000;
> SET disable_codegen=False;
> SET abort_on_error=1;
> SET exec_single_node_rows_threshold=0;
> SET mem_limit=2G;
> SET batch_size=0;
> SET num_nodes=0;
> -- executing against localhost:21000
> select * from functional.alltypesagg, (select 1) B limit 1;
>
> -- closing connection to: i<hostname>:21002
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org