You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2021/11/17 16:52:00 UTC
[jira] [Commented] (IMPALA-11006) Impalad crashes during query cancel tests
[ https://issues.apache.org/jira/browse/IMPALA-11006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17445356#comment-17445356 ]
ASF subversion and git services commented on IMPALA-11006:
----------------------------------------------------------
Commit 0cd5bb012856c364c2e7ec8962c01e488b718a8b in impala's branch refs/heads/master from Qifan Chen
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0cd5bb0 ]
IMPALA-11006: Impalad crashes during query cancel tests
This patch addresses impalad crash during query cancel tests for
CTAS. The root cause is that the wait thread in impalad assumes
that the coordinator or computation results are always available
once the worker thread async_exec_thread_ is joined. In reality,
this never happens as the query is cancelled.
The patch specifically checks the cancel status when it is found
that the coordinator is not available for CTAS and bails out
immediately. This prevents crash since the subsequent code that
assumes the coordinator or computation results are available is
never reached.
Testing:
1. Ran core tests successfully.
Change-Id: Ia49411f8525734b8d463d9ffbfbce705b90a8d73
Reviewed-on: http://gerrit.cloudera.org:8080/17999
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Impalad crashes during query cancel tests
> -----------------------------------------
>
> Key: IMPALA-11006
> URL: https://issues.apache.org/jira/browse/IMPALA-11006
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Qifan Chen
> Assignee: Qifan Chen
> Priority: Major
>
> The following stack trace was observed in a core generated during S3 build.
> {quote}Thread 485 (crashed)
> 0 libc-2.17.so + 0x351f7
> rax = 0x0000000000000000 rdx = 0x0000000000000006
> rcx = 0xffffffffffffffff rbx = 0x0000000000000004
> rsi = 0x0000000000006698 rdi = 0x000000000000144f
> rbp = 0x00007f89466b5220 rsp = 0x00007f89466b4ea8
> r8 = 0x0000000000000000 r9 = 0x00007f89466b4d20
> r10 = 0x0000000000000008 r11 = 0x0000000000000202
> r12 = 0x0000000007ea0100 r13 = 0x000000000000005a
> r14 = 0x0000000007ea0104 r15 = 0x0000000007e98720
> rip = 0x00007f8a65f091f7
> Found by: given as instruction pointer in context
> 1 impalad!google::LogMessage::Flush() + 0x1eb
> rbp = 0x00007f89466b5350 rsp = 0x00007f89466b5230
> rip = 0x00000000056aa19b
> Found by: previous frame's frame pointer
> 2 impalad!google::LogMessageFatal::~LogMessageFatal() + 0x9
> rbx = 0x0000000000000001 rbp = 0x00007f89466b53f0
> rsp = 0x00007f89466b52d0 r12 = 0x0000000007ea7638
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x00000000056add99
> Found by: call frame info
> 3 impalad!impala::ClientRequestState::SetCreateTableAsSelectResultSet() [client-request-state.cc : 1540 + 0xf]
> rbx = 0x0000000000000001 rbp = 0x00007f89466b53f0
> rsp = 0x00007f89466b52e0 r12 = 0x0000000007ea7638
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x000000000286fef1
> Found by: call frame info
> 4 impalad!impala::ClientRequestState::WaitInternal() [client-request-state.cc : 1083 + 0xf]
> rbx = 0x0000000000000001 rbp = 0x00007f89466b5550
> rsp = 0x00007f89466b5400 r12 = 0x00000000091cb7a0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x000000000286ab3b
> Found by: call frame info
> 5 impalad!impala::ClientRequestState::Wait() [client-request-state.cc : 1010 + 0x19]
> rbx = 0x00007f89466b5b28 rbp = 0x00007f89466b5640
> rsp = 0x00007f89466b5560 r12 = 0x00000000091cb7a0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x000000000286a030
> Found by: call frame info
> 6 impalad!boost::_mfi::mf0<void, impala::ClientRequestState>::operator()(impala::ClientRequestState*) const [mem_fn_template.hpp : 49 + 0x5]
> rbx = 0x00007f89466b5b28 rbp = 0x00007f89466b5660
> rsp = 0x00007f89466b5650 r12 = 0x00000000091cb7a0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x000000000287faed
> Found by: call frame info
> 7 impalad!void boost::_bi::list1<boost::_bi::value<impala::ClientRequestState*> >::operator()<boost::_mfi::mf0<void, impala::ClientRequestState>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, impala::ClientRequestState>&, boost::_bi::list0&, int) [bind.hpp : 259 + 0x35]
> rbx = 0x00007f89466b5b28 rbp = 0x00007f89466b56a0
> rsp = 0x00007f89466b5670 r12 = 0x00000000091cb7a0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x000000000287f259
> Found by: call frame info
> 8 impalad!boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ClientRequestState>, boost::_bi::list1<boost::_bi::value<impala::ClientRequestState*> > >::operator()() [bind.hpp : 1222 + 0x22]
> rbx = 0x0000000000006698 rbp = 0x00007f89466b56f0
> rsp = 0x00007f89466b56b0 r12 = 0x00000000091cb7a0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x000000000287e2f3
> Found by: call frame info
> 9 impalad!boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ClientRequestState>, boost::_bi::list1<boost::_bi::value<impala::ClientRequestState*> > >, void>::invoke(boost::detail::function::function_buffer&) [function_template.hpp : 159 + 0xc]
> rbx = 0x0000000000006698 rbp = 0x00007f89466b5720
> rsp = 0x00007f89466b5700 r12 = 0x00000000091cb7a0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x000000000287d07e
> Found by: call frame info
> 10 impalad!boost::function0<void>::operator()() const [function_template.hpp : 770 + 0x1d]
> rbx = 0x0000000000006698 rbp = 0x00007f89466b5760
> rsp = 0x00007f89466b5730 r12 = 0x00000000091cb7a0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x00000000022e55c0
> Found by: call frame info
> 11 impalad!impala::Thread::SuperviseThread(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) [thread.cc : 360 + 0xf]
> rbx = 0x0000000000006698 rbp = 0x00007f89466b5af0
> rsp = 0x00007f89466b5770 r12 = 0x00000000091cb7a0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x0000000002aab96a
> Found by: call frame info
> 12 impalad!void boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list0&, int) [bind.hpp : 531 + 0x15]
> rbx = 0x0000000002aab624 rbp = 0x00007f89466b5b60
> rsp = 0x00007f89466b5b00 r12 = 0x00007f89556dc7c0
> r13 = 0x00007f89556db190 r14 = 0x0000000016630d20
> r15 = 0x000000000000002c rip = 0x0000000002ab42cb
> Found by: call frame info
> 13 impalad!boost::_bi::bind_t<void, void (*)(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::list5<boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impala::PromiseMode)0>*> > >::operator()() [bind.hpp : 1222 + 0x22]
> rbx = 0x0000000016630b40 rbp = 0x00007f89466b5bb0
> rsp = 0x00007f89466b5b70 r12 = 0x0000000000000000
> r13 = 0x00007f89466b69c0 r14 = 0x00007f89466b6700
> r15 = 0x000000000000002c rip = 0x0000000002ab41ef
> Found by: call frame info{quote}
> The relevant section in the log file contains the following.
> {quote}I1102 04:17:23.806015 26053 Frontend.java:1637] 274523d21ab48e25:465bec8000000000] Analyzing query: create table ctas_cancel primary key (l_orderkey, l_partkey, l_suppkey, l_linenumber) partition by hash partitions 3 stored as kudu as select * from lineitem order by l_orderkey db: tpch_kudu
> ...
> F1102 04:17:32.149560 26264 client-request-state.cc:1540] Check failed: GetCoordinator() {quote}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org