You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Fang-Yu Rao (Jira)" <ji...@apache.org> on 2020/11/21 00:43:00 UTC
[jira] [Updated] (IMPALA-10348) test_sequential_startup_wait failed
due to query admission exceeding timeout 60000ms
[ https://issues.apache.org/jira/browse/IMPALA-10348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fang-Yu Rao updated IMPALA-10348:
---------------------------------
Description:
We found in a recent s3 build that the test of {{test_sequential_startup_wait}} failed due to admission for query exceeding timeout 60000ms.
The error message in the console output of the Jenkins job is the following.
{noformat}
Error Message:
E Query aborted:Admission for query exceeded timeout 60000ms in pool default-pool. Queued reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run. Additional Details: Not Applicable
{noformat}
Before the timeout was reached we saw the following entries in the corresponding log file produced by an impalad.
{noformat}
W1119 20:11:45.731312 20667 executor-group.cc:164] e64b11989b018148:20eb7ce500000000] Executor group default-pool-group1 is unhealthy: 1 out of 3 are available.
W1119 20:11:45.731338 20667 admission-controller.cc:1558] e64b11989b018148:20eb7ce500000000] Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
I1119 20:11:45.731348 20667 admission-controller.cc:1210] e64b11989b018148:20eb7ce500000000] Queuing, query id=e64b11989b018148:20eb7ce500000000 reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
I1119 20:11:45.773303 20040 admission-controller.cc:1876] Could not dequeue query id=e64b11989b018148:20eb7ce500000000 reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
{noformat}
The corresponding log files are also provided.
The test was recently revised in IMPALA-8830, maybe [~bikramjeet.vig] could provide some insight into it. Thanks!
was:
We found in a recent s3 build that the test of test_sequential_startup_wait failed due to admission for query exceeding timeout 60000ms.
The error message in the console output of the Jenkins job is the following.
{noformat}
Error Message:
E Query aborted:Admission for query exceeded timeout 60000ms in pool default-pool. Queued reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run. Additional Details: Not Applicable
{noformat}
Before the timeout was reached we saw the following entries in the corresponding log file produced by an impalad.
{noformat}
W1119 20:11:45.731312 20667 executor-group.cc:164] e64b11989b018148:20eb7ce500000000] Executor group default-pool-group1 is unhealthy: 1 out of 3 are available.
W1119 20:11:45.731338 20667 admission-controller.cc:1558] e64b11989b018148:20eb7ce500000000] Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
I1119 20:11:45.731348 20667 admission-controller.cc:1210] e64b11989b018148:20eb7ce500000000] Queuing, query id=e64b11989b018148:20eb7ce500000000 reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
I1119 20:11:45.773303 20040 admission-controller.cc:1876] Could not dequeue query id=e64b11989b018148:20eb7ce500000000 reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
{noformat}
The corresponding log files are also provided.
> test_sequential_startup_wait failed due to query admission exceeding timeout 60000ms
> ------------------------------------------------------------------------------------
>
> Key: IMPALA-10348
> URL: https://issues.apache.org/jira/browse/IMPALA-10348
> Project: IMPALA
> Issue Type: Bug
> Reporter: Fang-Yu Rao
> Assignee: Bikramjeet Vig
> Priority: Major
> Labels: broken-build, flaky
> Attachments: catalogd.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.ERROR.20201119-201129.19576, catalogd.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.INFO.20201119-201129.19576, catalogd.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.WARNING.20201119-201129.19576, impalad.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.ERROR.20201119-201129.19631, impalad.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.INFO.20201119-201129.19631, impalad.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.WARNING.20201119-201129.19631, statestored.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.ERROR.20201119-201129.19560, statestored.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.INFO.20201119-201129.19560, statestored.impala-ec2-centos74-m5-4xlarge-ondemand-0d33.vpc.cloudera.com.jenkins.log.WARNING.20201119-201129.19560
>
>
> We found in a recent s3 build that the test of {{test_sequential_startup_wait}} failed due to admission for query exceeding timeout 60000ms.
> The error message in the console output of the Jenkins job is the following.
> {noformat}
> Error Message:
> E Query aborted:Admission for query exceeded timeout 60000ms in pool default-pool. Queued reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run. Additional Details: Not Applicable
> {noformat}
> Before the timeout was reached we saw the following entries in the corresponding log file produced by an impalad.
> {noformat}
> W1119 20:11:45.731312 20667 executor-group.cc:164] e64b11989b018148:20eb7ce500000000] Executor group default-pool-group1 is unhealthy: 1 out of 3 are available.
> W1119 20:11:45.731338 20667 admission-controller.cc:1558] e64b11989b018148:20eb7ce500000000] Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
> I1119 20:11:45.731348 20667 admission-controller.cc:1210] e64b11989b018148:20eb7ce500000000] Queuing, query id=e64b11989b018148:20eb7ce500000000 reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
> I1119 20:11:45.773303 20040 admission-controller.cc:1876] Could not dequeue query id=e64b11989b018148:20eb7ce500000000 reason: Waiting for executors to start. Only DDL queries and queries scheduled only on the coordinator (either NUM_NODES set to 1 or when small query optimization is triggered) can currently run.
> {noformat}
> The corresponding log files are also provided.
> The test was recently revised in IMPALA-8830, maybe [~bikramjeet.vig] could provide some insight into it. Thanks!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org