You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (JIRA)" <ji...@apache.org> on 2019/03/23 23:39:00 UTC
[jira] [Commented] (IMPALA-6642) start-impala-cluster.py failing in
some custom cluster tests
[ https://issues.apache.org/jira/browse/IMPALA-6642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799843#comment-16799843 ]
ASF subversion and git services commented on IMPALA-6642:
---------------------------------------------------------
Commit fffbd2de073390b22441b04d99383cb5713a0dbe in impala's branch refs/heads/2.x from Taras Bobrovytsky
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=fffbd2d ]
IMPALA-6642 (Part 2): clean up start-impala-cluster.py
We clean up start-impala-cluster.py in general in this patch by using
logging instead of "print" and formatting strings using the format()
function. We make sure to include a timestamp in each log message in
order to make it easier to debug failures in custom cluster tests that
happen when starting the cluster.
Change-Id: I60169203c61ae6bc0a3ccd3dea355799b603efe5
Reviewed-on: http://gerrit.cloudera.org:8080/10780
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-on: http://gerrit.cloudera.org:8080/12798
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
> start-impala-cluster.py failing in some custom cluster tests
> ------------------------------------------------------------
>
> Key: IMPALA-6642
> URL: https://issues.apache.org/jira/browse/IMPALA-6642
> Project: IMPALA
> Issue Type: Bug
> Components: Infrastructure
> Affects Versions: Impala 2.12.0
> Reporter: Thomas Tauber-Marshall
> Priority: Critical
> Labels: broken-build
>
> Seen in recent builds, both on the 2.x branch. Not quite the same test/error, but they seem similar enough to probably be related:
> {noformat}
> 17:01:18 _________ ERROR at setup of TestAdmissionController.test_require_user __________
> 17:01:18 common/custom_cluster_test_suite.py:109: in setup_method
> 17:01:18 self._start_impala_cluster(cluster_args)
> 17:01:18 common/custom_cluster_test_suite.py:144: in _start_impala_cluster
> 17:01:18 check_call(cmd + options, close_fds=True)
> 17:01:18 /usr/lib64/python2.6/subprocess.py:505: in check_call
> 17:01:18 raise CalledProcessError(retcode, cmd)
> 17:01:18 E CalledProcessError: Command '['/data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/bin/start-impala-cluster.py', '--cluster_size=3', '--num_coordinators=3', '--log_dir=/data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/logs/custom_cluster_tests', '--log_level=1', '--impalad_args="-vmodule admission-controller=3 -fair_scheduler_allocation_path /data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/fe/src/test/resources/fair-scheduler-test2.xml -llama_site_path /data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/fe/src/test/resources/llama-site-test2.xml -disable_admission_control=false -require_username" ', '--state_store_args="-statestore_heartbeat_frequency_ms=100 -statestore_priority_update_frequency_ms=100" ']' returned non-zero exit status 1
> 17:01:18 ---------------------------- Captured stdout setup -----------------------------
> 17:01:18 Starting State Store logging to /data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 17:01:18 Starting Catalog Service logging to /data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 17:01:18 Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 17:01:18 Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 17:01:18 Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-2.x-core-asan/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 17:01:18 Error starting cluster: Expected 3 impalad(s), only 2 found
> 17:01:18
> 17:01:18 ---------------------------- Captured stderr setup -----------------------------
> 17:01:18 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es)
> 17:01:18 MainThread: Found 2 impalad/1 statestored/1 catalogd process(es)
> {noformat}
> {noformat}
> 16:42:41 _______ ERROR at setup of TestAuthorization.test_access_runtime_profile ________
> 16:42:41 common/custom_cluster_test_suite.py:109: in setup_method
> 16:42:41 self._start_impala_cluster(cluster_args)
> 16:42:41 common/custom_cluster_test_suite.py:144: in _start_impala_cluster
> 16:42:41 check_call(cmd + options, close_fds=True)
> 16:42:41 /usr/lib64/python2.6/subprocess.py:505: in check_call
> 16:42:41 raise CalledProcessError(retcode, cmd)
> 16:42:41 E CalledProcessError: Command '['/data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/bin/start-impala-cluster.py', '--cluster_size=3', '--num_coordinators=3', '--log_dir=/data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/logs/custom_cluster_tests', '--log_level=1', '--impalad_args="--server_name=server1 --authorization_policy_file=/test-warehouse/authz-policy.ini --authorized_proxy_user_config=hue=jenkins" ']' returned non-zero exit status 1
> 16:42:41 ---------------------------- Captured stdout setup -----------------------------
> 16:42:41 Starting State Store logging to /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/logs/custom_cluster_tests/statestored.INFO
> 16:42:41 Starting Catalog Service logging to /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/logs/custom_cluster_tests/catalogd.INFO
> 16:42:41 Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad.INFO
> 16:42:41 Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node1.INFO
> 16:42:41 Starting Impala Daemon logging to /data/jenkins/workspace/impala-asf-2.x-exhaustive/repos/Impala/logs/custom_cluster_tests/impalad_node2.INFO
> 16:42:41 Error starting cluster: num_known_live_backends did not reach expected value in time
> 16:42:41 ---------------------------- Captured stderr setup -----------------------------
> 16:42:41 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> 16:42:41 MainThread: Getting num_known_live_backends from ec2-m2-4xlarge-centos-6-4-0a73.vpc.cloudera.com:25000
> 16:42:41 MainThread: Debug webpage not yet available.
> 16:42:41 MainThread: Debug webpage not yet available.
> 16:42:41 MainThread: Waiting for num_known_live_backends=3. Current value: 0
> 16:42:41 MainThread: Getting num_known_live_backends from ec2-m2-4xlarge-centos-6-4-0a73.vpc.cloudera.com:25000
> 16:42:41 MainThread: Waiting for num_known_live_backends=3. Current value: 0
> 16:42:41 MainThread: Getting num_known_live_backends from ec2-m2-4xlarge-centos-6-4-0a73.vpc.cloudera.com:25000
> 16:42:41 MainThread: Waiting for num_known_live_backends=3. Current value: 1
> 16:42:41 MainThread: Getting num_known_live_backends from ec2-m2-4xlarge-centos-6-4-0a73.vpc.cloudera.com:25000
> 16:42:41 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 16:42:41 MainThread: Getting num_known_live_backends from ec2-m2-4xlarge-centos-6-4-0a73.vpc.cloudera.com:25000
> 16:42:41 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 16:42:41 MainThread: Getting num_known_live_backends from ec2-m2-4xlarge-centos-6-4-0a73.vpc.cloudera.com:25000
> 16:42:41 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 16:42:41 MainThread: Getting num_known_live_backends from ec2-m2-4xlarge-centos-6-4-0a73.vpc.cloudera.com:25000
> 16:42:41 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> ...
> 16:42:41 MainThread: Getting num_known_live_backends from ec2-m2-4xlarge-centos-6-4-0a73.vpc.cloudera.com:25000
> 16:42:41 MainThread: Waiting for num_known_live_backends=3. Current value: 2
> 16:42:41 MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org