You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Qifan Chen (Jira)" <ji...@apache.org> on 2021/04/20 17:24:00 UTC
[jira] [Created] (IMPALA-10666) AssertionError (Metric
impala-server.num-open-beeswax-sessions did not reach value 0 in 10s) seen
with ASAN test run
Qifan Chen created IMPALA-10666:
-----------------------------------
Summary: AssertionError (Metric impala-server.num-open-beeswax-sessions did not reach value 0 in 10s) seen with ASAN test run
Key: IMPALA-10666
URL: https://issues.apache.org/jira/browse/IMPALA-10666
Project: IMPALA
Issue Type: Test
Reporter: Qifan Chen
The following messages were seen with ASAN run #25 (https://master-03.jenkins.cloudera.com/job/impala-asf-master-core-asan/25/testReport/junit/shell.test_shell_interactive/TestImpalaShellInteractive/test_reconnect_table_format_and_file_extension____parquet_____parq_____protocol__beeswax_/). Mini core dumps were created too (see one of the stack traces below).
{code:java}
Error Message
AssertionError: Metric impala-server.num-open-beeswax-sessions did not reach value 0 in 10s. Dumping debug webpages in JSON format... Dumped memz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/memz.json Dumped metrics JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/metrics.json Dumped queries JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/queries.json Dumped sessions JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/sessions.json Dumped threadz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/threadz.json Dumped rpcz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/rpcz.json Dumping minidumps for impalads/catalogds... Dumped minidump for Impalad PID 11246 Dumped minidump for Impalad PID 11249 Dumped minidump for Impalad PID 11255 Dumped minidump for Catalogd PID 11157
Stacktrace
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/shell/test_shell_interactive.py:421: in test_reconnect
"first impalad should not have any remaining open sessions.")
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/shell/test_shell_interactive.py:183: in _wait_for_num_open_sessions
actual = impala_service.wait_for_metric_value(metric_name, expected)
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_service.py:136: in wait_for_metric_value
self.__metric_timeout_assert(metric_name, expected_value, timeout)
/data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/tests/common/impala_service.py:203: in __metric_timeout_assert
assert 0, assert_string
E AssertionError: Metric impala-server.num-open-beeswax-sessions did not reach value 0 in 10s.
E Dumping debug webpages in JSON format...
E Dumped memz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/memz.json
E Dumped metrics JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/metrics.json
E Dumped queries JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/queries.json
E Dumped sessions JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/sessions.json
E Dumped threadz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/threadz.json
E Dumped rpcz JSON to $IMPALA_HOME/logs/metric_timeout_diags_20210419_23:35:43/json/rpcz.json
E Dumping minidumps for impalads/catalogds...
E Dumped minidump for Impalad PID 11246
E Dumped minidump for Impalad PID 11249
E Dumped minidump for Impalad PID 11255
E Dumped minidump for Catalogd PID 11157
{code}
{code:java}
Error Message
Minidump generated: /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/logs/ee_tests/minidumps/catalogd/74546e5b-99cb-4682-a597c2bc-71978a6d.dmp
Standard Error
Operating system: Linux
0.0.0 Linux 3.10.0-693.5.2.el7.x86_64 #1 SMP Fri Oct 20 20:32:50 UTC 2017 x86_64
CPU: amd64
family 6 model 85 stepping 7
1 CPU
GPU: UNKNOWN
Crash reason: DUMP_REQUESTED
Crash address: 0x5bb0747
Process uptime: not available
Thread 0 (crashed)
0 impalad!google_breakpad::ExceptionHandler::WriteMinidump() + 0x57
rax = 0x0000000005d9a723 rdx = 0x0000000000000000
rcx = 0x0000000005bb072f rbx = 0x0000000000000000
rsi = 0x0000000000000001 rdi = 0x00007fff2bc0fd38
rbp = 0x00007fff2bc104b0 rsp = 0x00007fff2bc0fca0
r8 = 0x0000000000000000 r9 = 0x0000000000000000
r10 = 0x0000000000000018 r11 = 0x0000000000000000
r12 = 0x0000000009a364e0 r13 = 0x00007fff2bc10390
r14 = 0x0000000002e770e0 r15 = 0x0000000000000000
rip = 0x0000000005bb0747
Found by: given as instruction pointer in context
1 impalad!google_breakpad::ExceptionHandler::WriteMinidump(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool (*)(google_breakpad::MinidumpDescriptor const&, void*, bool), void*) + 0x15c
rbx = 0x00007fff2bc10300 rbp = 0x00007fff2bc104b0
rsp = 0x00007fff2bc102f0 r12 = 0x0000000009a364e0
r13 = 0x00007fff2bc10390 r14 = 0x0000000002e770e0
r15 = 0x0000000000000000 rip = 0x0000000005bb109c
Found by: call frame info
2 libpthread-2.17.so + 0xf5e0
rbx = 0x00007fff2bc112c0 rbp = 0x00007fff2bc11360
rsp = 0x00007fff2bc104c0 r12 = 0x00006160000e28d8
r13 = 0x00006160000e2900 r14 = 0x00007fff2bc111e0
r15 = 0x00000fffe578223c rip = 0x00007f71068cc5e0
Found by: call frame info
3 impalad!boost::thread::join_noexcept() + 0x54
rbp = 0x00007fff2bc11380 rsp = 0x00007fff2bc11370
rip = 0x000000000485a854
Found by: previous frame's frame pointer
4 impalad!boost::thread::join() [thread.hpp : 766 + 0x8]
rbx = 0x00007fff2bc11480 rbp = 0x00007fff2bc114f0
rsp = 0x00007fff2bc113c0 r12 = 0x0000602000276e10
r13 = 0x000010006577a278 r14 = 0x00000fffe5782278
rip = 0x00000000025ef348
Found by: call frame info
5 impalad!impala::ThriftServer::Join() [thrift-server.cc : 465 + 0x8]
rbx = 0x00007fff2bc11580 rbp = 0x00007fff2bc115f0
rsp = 0x00007fff2bc11500 r12 = 0x00006140000af840
r13 = 0x00006140000af928 r14 = 0x00000fffe57822a0
r15 = 0x0000000000000000 rip = 0x00000000025e9d7c
Found by: call frame info
6 impalad!CatalogdMain(int, char**) [catalogd-main.cc : 88 + 0x5]
rbx = 0x00007fff2bc11d80 rbp = 0x00007fff2bc11e30
rsp = 0x00007fff2bc11600 r12 = 0x00007fff2bc11600
r13 = 0x00000fffe5782340 r14 = 0x000010006577a2c0
r15 = 0x00007fff2bc11d50 rip = 0x0000000001cb509d
Found by: call frame info
7 impalad!main [daemon-main.cc : 42 + 0xc]
rbx = 0x00007fff2bc11f20 rbp = 0x00007fff2bc11fa0
rsp = 0x00007fff2bc11e40 r12 = 0x00007fff2bc11e40
r13 = 0x00007fff2bc11ea0 r14 = 0x00007fff2bc11ea0
r15 = 0x000010006577a3c8 rip = 0x0000000001cb33fa
Found by: call frame info
8 libc-2.17.so + 0x21c05
rbx = 0x0000000000000000 rbp = 0x0000000000000000
rsp = 0x00007fff2bc11fb0 r12 = 0x0000000001bbc0be
r13 = 0x00007fff2bc12080 r14 = 0x0000000000000000
r15 = 0x0000000000000000 rip = 0x00007f7103237c05
Found by: call frame info
9 impalad + 0x18b3250
rsp = 0x00007fff2bc11fd0 rip = 0x0000000001cb3250
Found by: stack scanning
10 impalad!_GLOBAL__sub_I_json_escaping.cc + 0x2e
rsp = 0x00007fff2bc11fe8 rip = 0x0000000001bbc0be
Found by: stack scanning
11 0x7fff2bc12080
rsp = 0x00007fff2bc11ff0 rip = 0x00007fff2bc12080
Found by: call frame info
12 ld-2.17.so + 0xf47f
rsp = 0x00007fff2bc12030 rip = 0x00007f7106ceb47f
Found by: stack scanning
13 impalad!_GLOBAL__sub_I_json_escaping.cc + 0x2e
rsp = 0x00007fff2bc12058 rip = 0x0000000001bbc0be
Found by: stack scanning
14 0x7fff2bc12080
rsp = 0x00007fff2bc12060 rip = 0x00007fff2bc12080
Found by: call frame info
15 impalad!_start + 0x29
rsp = 0x00007fff2bc12070 rip = 0x0000000001bbc0e7
Found by: stack scanning
16 0x7fff2bc12078
rsp = 0x00007fff2bc12078 rip = 0x00007fff2bc12078
Found by: call frame info
Thread 1
0 libc-2.17.so + 0xbf1ad
rax = 0xfffffffffffffdfc rdx = 0x0000000000000000
rcx = 0xffffffffffffffff rbx = 0x00007f70fea66160
rsi = 0x00007f70fea66150 rdi = 0x00007f70fea66150
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org