You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tianyi Wang (JIRA)" <ji...@apache.org> on 2017/10/25 00:25:00 UTC
[jira] [Resolved] (IMPALA-6060) Crash in JniUtfCharGuard::create()
[ https://issues.apache.org/jira/browse/IMPALA-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tianyi Wang resolved IMPALA-6060.
---------------------------------
Resolution: Fixed
Fix Version/s: Impala 2.11.0
IMPALA-6060: Check the return value of JNI exception handling functions
When JVM runs out of memory and throws an error to JNI, the error
handling code uses JNI to get the exception message, resulting in a
null pointer and crashing the process. This patch adds error handling
code to JniUtil::GetJniExceptionMsg().
Change-Id: Ie3ed88bf8739c56a066f2402727c8204e96aa116
Reviewed-on: http://gerrit.cloudera.org:8080/8334
Reviewed-by: Alex Behm <al...@cloudera.com>
Tested-by: Impala Public Jenkins
> Crash in JniUtfCharGuard::create()
> ----------------------------------
>
> Key: IMPALA-6060
> URL: https://issues.apache.org/jira/browse/IMPALA-6060
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.10.0
> Reporter: Tim Armstrong
> Assignee: Tianyi Wang
> Priority: Blocker
> Labels: crash
> Fix For: Impala 2.11.0
>
>
> {noformat}
> Operating system: Linux
> 0.0.0 Linux 2.6.32-358.14.1.el6.x86_64 #1 SMP Tue Jul 16 23:51:20 UTC 2013 x86_64
> CPU: amd64
> family 6 model 45 stepping 7
> 24 CPUs
> GPU: UNKNOWN
> Crash reason: SIGSEGV
> Crash address: 0x0
> Process uptime: not available
> Thread 1281 (crashed)
> 0 libjvm.so + 0x6e1548
> rax = 0x00007f3be7fddaf0 rdx = 0x000000019b4621f8
> rcx = 0x00007f377f5ea000 rbx = 0x0000000000000000
> rsi = 0x000000005c4f1000 rdi = 0x000000005c4f1000
> rbp = 0x00007f377f5e9f60 rsp = 0x00007f377f5e9ec0
> r8 = 0x0000000000000001 r9 = 0x0000000000000000
> r10 = 0x0000000000000001 r11 = 0x00007f3bd70ea9e0
> r12 = 0x000000005c4f1000 r13 = 0x0000000000000000
> r14 = 0x00007f377f5ea000 r15 = 0x00007f377f5e9f00
> rip = 0x00007f3be7709548
> Found by: given as instruction pointer in context
> 1 impalad!impala::JniUtfCharGuard::create(JNIEnv_*, _jstring*, impala::JniUtfCharGuard*) [jni.h : 1617 + 0xc]
> rbp = 0x00007f377f5ea0b0 rsp = 0x00007f377f5e9f70
> rip = 0x0000000000b70d79
> Found by: previous frame's frame pointer
> 2 impalad!impala::JniUtil::GetJniExceptionMsg(JNIEnv_*, bool, std::string const&) [jni-util.cc : 190 + 0x5]
> rbx = 0x000000005c4f11f8 rbp = 0x00007f377f5ea0b0
> rsp = 0x00007f377f5e9fc0 r12 = 0x00007f377f5ea340
> r13 = 0x00007f377f5ea050 r14 = 0x00007f377f5ea0c0
> rip = 0x0000000000b7160e
> Found by: call frame info
> 3 impalad!impala::Status impala::SerializeThriftMsg<impala::TUpdateCatalogCacheRequest const>(JNIEnv_*, impala::TUpdateCatalogCacheRequest const*, _jbyteArray**) [jni-thrift-util.h : 40 + 0x13]
> rbx = 0x000000005c4f11f8 rbp = 0x00007f377f5ea340
> rsp = 0x00007f377f5ea0c0 r12 = 0x00007f377f5ea350
> r13 = 0x00007f377f5ea0f0 r14 = 0x000000001f73ecca
> r15 = 0x00007f35956f4000 rip = 0x0000000000a7d322
> Found by: call frame info
> 4 impalad!impala::Frontend::UpdateCatalogCache(std::vector<impala::TUpdateCatalogCacheRequest, std::allocator<impala::TUpdateCatalogCacheRequest> > const&, impala::TUpdateCatalogCacheResponse*) [jni-util.h : 331 + 0x5]
> rbx = 0x000000005c4f11f8 rbp = 0x0000000000000000
> rsp = 0x00007f377f5ea2c0 r12 = 0x00007f377f5ea4e0
> r13 = 0x00007f377f5ea350 r14 = 0x00000000028a4169
> r15 = 0x00007f377f5ea470 rip = 0x0000000000a7a1ce
> Found by: call frame info
> 5 impalad!impala::ImpalaServer::CatalogUpdateCallback(std::map<std::string, impala::TTopicDelta, std::less<std::string>, std::allocator<std::pair<std::string const, impala::TTopicDelta> > > const&, std::vector<impala::TTopicDelta, std::allocator<impala::TTopicDelta> >*) [impala-server.cc : 1419 + 0xc]
> rbx = 0x00007f377f5ea520 rbp = 0x0000000000000000
> rsp = 0x00007f377f5ea3a0 r12 = 0x00007f377f5ea470
> r13 = 0x00007f377f5ead90 r14 = 0x00000000028a4169
> r15 = 0x0000000000000000 rip = 0x0000000000a94559
> Found by: call frame info
> 6 impalad!impala::StatestoreSubscriber::UpdateState(std::map<std::string, impala::TTopicDelta, std::less<std::string>, std::allocator<std::pair<std::string const, impala::TTopicDelta> > > const&, impala::TUniqueId const&, std::vector<impala::TTopicDelta, std::allocator<impala::TTopicDelta> >*, bool*) [function_template.hpp : 767 + 0xa]
> rbx = 0x00000000097486a0 rbp = 0x00007f377f5eb890
> rsp = 0x00007f377f5eb640 r12 = 0x00000000097486a0
> r13 = 0x000000000a60e700 r14 = 0x015edf1e8faf89f6
> r15 = 0x000000000275cce8 rip = 0x0000000000b2fea3
> Found by: call frame info
> 7 impalad!impala::StatestoreSubscriberThriftIf::UpdateState(impala::TUpdateStateResponse&, impala::TUpdateStateRequest const&) [statestore-subscriber.cc : 87 + 0x20]
> rbx = 0x00007f377f5eb858 rbp = 0x00007f377f5eb780
> rsp = 0x00007f377f5eb780 r12 = 0x00000000ffffffff
> r13 = 0x0000000000000000 r14 = 0x0000000009608940
> r15 = 0x00007f377f5eb850 rip = 0x0000000000b357b3
> Found by: call frame info
> 8 impalad!impala::StatestoreSubscriberProcessor::process_UpdateState(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) [StatestoreSubscriber.cpp : 501 + 0x6]
> rbx = 0x000000000977ca80 rbp = 0x00000001135957d0
> rsp = 0x00007f377f5eb7d0 r12 = 0x00000000ffffffff
> r13 = 0x0000000000000000 r14 = 0x0000000009608940
> r15 = 0x00007f377f5eb850 rip = 0x0000000000ddad68
> Found by: call frame info
> 9 impalad!impala::StatestoreSubscriberProcessor::dispatchCall(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::string const&, int, void*) [StatestoreSubscriber.cpp : 474 + 0x15]
> rbx = 0x000000022209a768 rbp = 0x0000000009758ec0
> rsp = 0x00007f377f5eb970 r12 = 0x0000000009608940
> r13 = 0x0000000000000000 r14 = 0x0000000000000009
> r15 = 0x000000000000000b rip = 0x0000000000dd9fa4
> Found by: call frame info
> 10 impalad!apache::thrift::TDispatchProcessor::process(boost::shared_ptr<apache::thrift::protocol::TProtocol>, boost::shared_ptr<apache::thrift::protocol::TProtocol>, void*) [TDispatchProcessor.h : 121 + 0x1c]
> rbx = 0x00007f3be682c7a0 rbp = 0x00007f377f5ebad0
> rsp = 0x00007f377f5eba30 r12 = 0x00007f377f5eba50
> r13 = 0x0000000009608940 r14 = 0x00007f377f5ebae0
> r15 = 0x00000000f020bf40 rip = 0x000000000080e4dc
> Found by: call frame info
> 11 impalad!apache::thrift::server::TAcceptQueueServer::Task::run() [TAcceptQueueServer.cpp : 77 + 0x11]
> rbx = 0x00000000a06c7d80 rbp = 0x00007f377f5ebae0
> rsp = 0x00007f377f5ebaa0 r12 = 0x00007f377f5ebad0
> r13 = 0x00007f377f5ebaf0 r14 = 0x00007f377f5ebaf0
> r15 = 0x00000000f020bf40 rip = 0x00000000009d9f3f
> Found by: call frame info
> 12 impalad!impala::ThriftThread::RunRunnable(boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*) [thrift-thread.cc : 74 + 0x6]
> rbx = 0x00007f3b5da9b8f0 rbp = 0x00007f377f5ebb80
> rsp = 0x00007f377f5ebb40 r12 = 0x00007f377f5ec700
> r13 = 0x0000000000001053 r14 = 0x00007f377f5ebbf0
> r15 = 0x00007f3be682c7a0 rip = 0x00000000009d4559
> Found by: call frame info
> 13 impalad!boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf2<void, impala::ThriftThread, boost::shared_ptr<apache::thrift::concurrency::Runnable>, impala::Promise<unsigned long>*>, boost::_bi::list3<boost::_bi::value<impala::ThriftThread*>, boost::_bi::value<boost::shared_ptr<apache::thrift::concurrency::Runnable> >, boost::_bi::value<impala::Promise<unsigned long>*> > >, void>::invoke(boost::detail::function::function_buffer&) [mem_fn_template.hpp : 280 + 0xf]
> rbx = 0x00007f377f5ebb80 rbp = 0x00007f377f5ebca0
> rsp = 0x00007f377f5ebb70 r12 = 0x0000000003727f80
> r13 = 0x0000000000001053 r14 = 0x00007f377f5ebbf0
> r15 = 0x00007f3be682c7a0 rip = 0x00000000009d5332
> Found by: call frame info
> 14 impalad!impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*) [function_template.hpp : 767 + 0x7]
> rbx = 0x00007f3b5da9b660 rbp = 0x00007f377f5ebca0
> rsp = 0x00007f377f5ebbb0 r12 = 0x0000000003727f80
> r13 = 0x0000000000001053 r14 = 0x00007f377f5ebbf0
> r15 = 0x00007f3be682c7a0 rip = 0x0000000000bd3eb2
> Found by: call frame info
> 15 impalad!boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::Promise<long>*), boost::_bi::list4<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::Promise<long>*> > > >::run() [bind.hpp : 457 + 0x6]
> rbx = 0x00000000e568b800 rbp = 0x00007f377f5ebcc0
> rsp = 0x00007f377f5ebcb0 r12 = 0x0000000000bd3cf0
> r13 = 0x00007f377f5ec9c0 r14 = 0x000000018cb31190
> r15 = 0x0000000000000003 rip = 0x0000000000bd4614
> Found by: call frame info
> 16 impalad!thread_proxy + 0xda
> rbx = 0x0000000000000000 rbp = 0x0000000000000000
> rsp = 0x00007f377f5ebd00 r12 = 0x000000365261c360
> r13 = 0x00007f377f5ec9c0 r14 = 0x000000018cb31190
> r15 = 0x0000000000000003 rip = 0x0000000000e607fa
> Found by: call frame info
> 17 libpthread-2.12.so + 0x7aa1
> rbx = 0x0000000000000000 rbp = 0x0000000000000000
> rsp = 0x00007f377f5ebd40 r12 = 0x000000365261c360
> r13 = 0x00007f377f5ec9c0 r14 = 0x000000018cb31190
> r15 = 0x0000000000000003 rip = 0x0000003652407aa1
> Found by: call frame info
> 18 libc-2.12.so + 0xe893d
> rsp = 0x00007f377f5ebe80 rip = 0x00000036520e893d
> Found by: stack scanning
> {noformat}
> We saw the above crash on an internal cluster.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)