You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Henry Robinson (JIRA)" <ji...@apache.org> on 2017/09/01 17:17:00 UTC
[jira] [Resolved] (IMPALA-5816) ssl-related custom cluster tests
failing during setup on exhaustive RHEL7
[ https://issues.apache.org/jira/browse/IMPALA-5816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Henry Robinson resolved IMPALA-5816.
------------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.11.0
https://github.com/apache/incubator-impala/commit/c163ac1468e4d878c3516ec933c69fb66851af01
> ssl-related custom cluster tests failing during setup on exhaustive RHEL7
> -------------------------------------------------------------------------
>
> Key: IMPALA-5816
> URL: https://issues.apache.org/jira/browse/IMPALA-5816
> Project: IMPALA
> Issue Type: Bug
> Components: Security
> Affects Versions: Impala 2.10.0
> Reporter: David Knupp
> Assignee: Henry Robinson
> Priority: Critical
> Fix For: Impala 2.11.0
>
>
> Tests that were seen to fail include:
> * TestClientSsl.test_tls_v12
> * TestClientSsl.test_wildcard_ssl
> * TestClientSsl.test_wildcard_san_ssl
> Example stack trace:
> {noformat}
> self = <test_client_ssl.TestClientSsl object at 0x5bf1090>
> method = <bound method TestClientSsl.test_tls_v12 of <test_client_ssl.TestClientSsl object at 0x5bf1090>>
> def setup_method(self, method):
> cluster_args = list()
> for arg in [IMPALAD_ARGS, STATESTORED_ARGS, CATALOGD_ARGS]:
> if arg in method.func_dict:
> cluster_args.append("--%s=\"%s\" " % (arg, method.func_dict[arg]))
> # Start a clean new cluster before each test
> > self._start_impala_cluster(cluster_args)
> common/custom_cluster_test_suite.py:103:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> common/custom_cluster_test_suite.py:129: in _start_impala_cluster
> check_call(cmd + options, close_fds=True)
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> popenargs = (['/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/bin/start-impala-cluster.py', '--cluster_si...cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcardCA.pem --ssl_cipher_list=AES128-GCM-SHA256 " ', ...],)
> kwargs = {'close_fds': True}, retcode = 1
> cmd = ['/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/bin/start-impala-cluster.py', '--cluster_siz...a-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcardCA.pem --ssl_cipher_list=AES128-GCM-SHA256 " ', ...]
> def check_call(*popenargs, **kwargs):
> """Run command with arguments. Wait for command to complete. If
> the exit code was zero then return, otherwise raise
> CalledProcessError. The CalledProcessError object will have the
> return code in the returncode attribute.
>
> The arguments are the same as for the Popen constructor. Example:
>
> check_call(["ls", "-l"])
> """
> retcode = call(*popenargs, **kwargs)
> if retcode:
> cmd = kwargs.get("args")
> if cmd is None:
> cmd = popenargs[0]
> > raise CalledProcessError(retcode, cmd)
> E CalledProcessError: Command '['/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/bin/start-impala-cluster.py', '--cluster_size=3', '--num_coordinators=3', '--log_dir=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/logs/custom_cluster_tests', '--log_level=1', '--impalad_args="--ssl_server_certificate=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcard-cert.pem --ssl_private_key=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcard-cert.key --ssl_minimum_version=tlsv1.2 --ssl_client_ca_certificate=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcardCA.pem --ssl_cipher_list=AES128-GCM-SHA256 " ', '--state_store_args="--ssl_server_certificate=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcard-cert.pem --ssl_private_key=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcard-cert.key --ssl_minimum_version=tlsv1.2 --ssl_client_ca_certificate=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcardCA.pem --ssl_cipher_list=AES128-GCM-SHA256 " ', '--catalogd_args="--ssl_server_certificate=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcard-cert.pem --ssl_private_key=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcard-cert.key --ssl_minimum_version=tlsv1.2 --ssl_client_ca_certificate=/data/jenkins/workspace/impala-cdh5-trunk-exhaustive-rhel7/repos/Impala/be/src/testutil/wildcardCA.pem --ssl_cipher_list=AES128-GCM-SHA256 " ']' returned non-zero exit status 1
> {noformat}
> Standard error output:
> {noformat}
> MainThread: Found 3 impalad/1 statestored/1 catalogd process(es)
> MainThread: Getting num_known_live_backends from impala-boost-static-burst-slave-el7-1b82.vpc.cloudera.com:25000
> MainThread: Debug webpage not yet available.
> MainThread: Waiting for num_known_live_backends=3. Current value: 0
> MainThread: Getting num_known_live_backends from impala-boost-static-burst-slave-el7-1b82.vpc.cloudera.com:25000
> MainThread: Waiting for num_known_live_backends=3. Current value: 0
> MainThread: Getting num_known_live_backends from impala-boost-static-burst-slave-el7-1b82.vpc.cloudera.com:25000
> MainThread: Waiting for num_known_live_backends=3. Current value: 0
> MainThread: Getting num_known_live_backends from impala-boost-static-burst-slave-el7-1b82.vpc.cloudera.com:25000
> MainThread: Waiting for num_known_live_backends=3. Current value: 0
> etc...
> etc...
> {noformat}
> When I look at the impalad.INFO log, this entry shows up at that timestamp:
> {noformat}
> I0818 00:43:29.000936 17712 Frontend.java:888] Compiling query: drop role grant_revoke_test_admin
> I0818 00:43:29.001284 17712 Frontend.java:927] Compiled query.
> I0818 00:43:29.021884 17712 impala-beeswax-server.cc:189] get_results_metadata(): query_id=114d057e65f76f3d:60e4c38f00000000
> I0818 00:43:29.316861 17712 status.cc:122] Query id 114d057e65f76f3d:60e4c38f00000000 not found.
> @ 0x12574e3 impala::Status::Status()
> @ 0x1512d22 impala::ImpalaServer::GetExecSummary()
> @ 0x158594a impala::ImpalaServer::GetExecSummary()
> @ 0x1a595d5 impala::ImpalaServiceProcessor::process_GetExecSummary()
> @ 0x1a56619 impala::ImpalaServiceProcessor::dispatchCall()
> @ 0x11fbb96 apache::thrift::TDispatchProcessor::process()
> @ 0x29076db apache::thrift::server::TThreadPoolServer::Task::run()
> @ 0x28efed9 apache::thrift::concurrency::ThreadManager::Worker::run()
> @ 0x13c7a6f impala::ThriftThread::RunRunnable()
> @ 0x13c919b boost::_mfi::mf2<>::operator()()
> @ 0x13c9031 boost::_bi::list3<>::operator()<>()
> @ 0x13c8d7d boost::_bi::bind_t<>::operator()()
> @ 0x13c8c90 boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x13d68ba boost::function0<>::operator()()
> @ 0x16a1b3d impala::Thread::SuperviseThread()
> @ 0x16aa50c boost::_bi::list4<>::operator()<>()
> @ 0x16aa44f boost::_bi::bind_t<>::operator()()
> @ 0x16aa412 boost::detail::thread_data<>::run()
> @ 0x1b8606a thread_proxy
> @ 0x7f8d4de4edf3 start_thread
> @ 0x7f8d4db7c1ad __clone
> I0818 00:43:29.318055 17712 impala-beeswax-server.cc:231] close(): query_id=114d057e65f76f3d:60e4c38f00000000
> I0818 00:43:29.318068 17712 impala-server.cc:991] UnregisterQuery(): query_id=114d057e65f76f3d:60e4c38f00000000
> I0818 00:43:29.318078 17712 impala-server.cc:1077] Cancel(): query_id=114d057e65f76f3d:60e4c38f00000000
> I0818 00:43:29.319111 17712 impala-server.cc:1791] Connection from client ::ffff:127.0.0.1:36947 closed, closing 1 associated session(s)
> I0818 00:43:29.587154 17712 status.cc:122] Session closed
> @ 0x12574e3 impala::Status::Status()
> @ 0x151835f impala::ImpalaServer::CloseSessionInternal()
> @ 0x151e894 impala::ImpalaServer::ConnectionEnd()
> @ 0x13bb2e4 impala::ThriftServer::ThriftServerEventProcessor::deleteContext()
> @ 0x29081b3 apache::thrift::server::TThreadPoolServer::Task::run()
> @ 0x28efed9 apache::thrift::concurrency::ThreadManager::Worker::run()
> @ 0x13c7a6f impala::ThriftThread::RunRunnable()
> @ 0x13c919b boost::_mfi::mf2<>::operator()()
> @ 0x13c9031 boost::_bi::list3<>::operator()<>()
> @ 0x13c8d7d boost::_bi::bind_t<>::operator()()
> @ 0x13c8c90 boost::detail::function::void_function_obj_invoker0<>::invoke()
> @ 0x13d68ba boost::function0<>::operator()()
> @ 0x16a1b3d impala::Thread::SuperviseThread()
> @ 0x16aa50c boost::_bi::list4<>::operator()<>()
> @ 0x16aa44f boost::_bi::bind_t<>::operator()()
> @ 0x16aa412 boost::detail::thread_data<>::run()
> @ 0x1b8606a thread_proxy
> @ 0x7f8d4de4edf3 start_thread
> @ 0x7f8d4db7c1ad __clone
> I0818 00:43:29.596240 17731 impala-server.cc:1391] Catalog topic entry deletion: ROLE:grant_revoke_test_admin
> I0818 00:43:29.596282 17731 impala-server.cc:1391] Catalog topic entry deletion: PRIVILEGE:4.server=server1
> {noformat}
> {noformat}
> Query (id=114d057e65f76f3d:60e4c38f00000000)
> DEBUG MODE WARNING: Query profile created while running a DEBUG build of Impala. Use RELEASE builds to measure query performance.
> Summary
> Session ID: ab4fcedc41970d96:a677076a3a30f086
> Session Type: BEESWAX
> Start Time: 2017-08-18 00:43:29.000313000
> End Time: 2017-08-18 00:43:29.318100000
> Query Type: DDL
> Query State: FINISHED
> Query Status: OK
> Impala Version: impalad version 2.10.0-SNAPSHOT DEBUG (build 68419b2814b5633fa125c456d61cd8f3316f39e9)
> User: jenkins
> Connected User: jenkins
> Delegated User:
> Network Address: ::ffff:127.0.0.1:36947
> Default Db: default
> Sql Statement: drop role grant_revoke_test_admin
> Coordinator: impala-boost-static-burst-slave-el7-1b82.vpc.cloudera.com:22000
> Query Options (non default): MT_DOP=0
> DDL Type: DROP_ROLE
>
> Query Timeline
> Query submitted: 0
> Planning finished: 1000012
> Request finished: 21000258
> First row fetched: 22000270
> Unregister query: 318003909
> ImpalaServer
> - CatalogOpExecTimer: 15000184
> - ClientFetchWaitTimer: 297003651
> - InactiveTotalTime: 0
> - RowMaterializationTimer: 0
> - TotalTime: 0
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)