You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Tim Armstrong (Jira)" <ji...@apache.org> on 2020/05/19 02:50:00 UTC
[jira] [Resolved] (IMPALA-9730) TSAN data race in
RuntimeFilterBank::CancelLocked()
[ https://issues.apache.org/jira/browse/IMPALA-9730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tim Armstrong resolved IMPALA-9730.
-----------------------------------
Fix Version/s: Impala 4.0
Resolution: Fixed
> TSAN data race in RuntimeFilterBank::CancelLocked()
> ---------------------------------------------------
>
> Key: IMPALA-9730
> URL: https://issues.apache.org/jira/browse/IMPALA-9730
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Sahil Takiar
> Assignee: Tim Armstrong
> Priority: Major
> Fix For: Impala 4.0
>
>
> I started seeing the issue while working on IMPALA-9199. I don't see a reason why IMPALA-9199 would have started causing this though. The failure comes up in expr-test which should be unrelated to query retries.
> {code:java}
> WARNING: ThreadSanitizer: data race (pid=29921)
> Read of size 1 at 0x7b340005c5c8 by thread T370:
> #0 impala::RuntimeFilterBank::CancelLocked() /home/stakiar/Impala/be/src/runtime/runtime-filter-bank.cc:435 (unifiedbetests+0x2da33cb)
> #1 impala::RuntimeFilterBank::Close() /home/stakiar/Impala/be/src/runtime/runtime-filter-bank.cc:459 (unifiedbetests+0x2da371a)
> #2 impala::QueryState::ReleaseBackendResources() /home/stakiar/Impala/be/src/runtime/query-state.cc:115 (unifiedbetests+0x2d7bcbb)
> #3 impala::QueryState::ReleaseBackendResourceRefcount() /home/stakiar/Impala/be/src/runtime/query-state.cc:731 (unifiedbetests+0x2d8084f)
> #4 impala::QueryExecMgr::ExecuteQueryHelper(impala::QueryState*) /home/stakiar/Impala/be/src/runtime/query-exec-mgr.cc:157 (unifiedbetests+0x2df74dd)
> #5 boost::_mfi::mf1<void, impala::QueryExecMgr, impala::QueryState*>::operator()(impala::QueryExecMgr*, impala::QueryState*) const /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/mem
> _fn_template.hpp:165 (unifiedbetests+0x2df87bd)
> #6 void boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*>, boost::_bi::value<impala::QueryState*> >::operator()<boost::_mfi::mf1<void, impala::QueryExecMgr, impala::QueryState*>, boost::_bi::
> list0>(boost::_bi::type<void>, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::QueryState*>&, boost::_bi::list0&, int) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:319 (
> unifiedbetests+0x2df86fd)
> #7 boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::QueryState*>, boost::_bi::list2<boost::_bi::value<impala::QueryExecMgr*>, boost::_bi::value<impala::QueryState*> > >::o
> perator()() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222 (unifiedbetests+0x2df8663)
> #8 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf1<void, impala::QueryExecMgr, impala::QueryState*>, boost::_bi::list2<boost::_bi::value<impala::QueryExec
> Mgr*>, boost::_bi::value<impala::QueryState*> > >, void>::invoke(boost::detail::function::function_buffer&) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/function/function_template.hpp:159
> (unifiedbetests+0x2df8401)
> #9 boost::function0<void>::operator()() const /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/function/function_template.hpp:770 (unifiedbetests+0x2bc2061)
> #10 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /home/stakiar/Impa
> la/be/src/util/thread.cc:360 (unifiedbetests+0x3533c86)
> #11 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<
> impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::Prom
> iseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseM
> ode)0>*), boost::_bi::list0&, int) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:531 (unifiedbetests+0x353be7c)
> #12 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::lis
> t5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impal
> a::PromiseMode)0>*> > >::operator()() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222 (unifiedbetests+0x353bd93)
> #13 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::Promis
> eMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<i
> mpala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/thread/detail/thread.hpp:116 (unifiedbetests+0x353ba80)
> #14 thread_proxy thread.cpp:? (unifiedbetests+0x37c7869)
> Previous write of size 1 at 0x7b340005c5c8 by thread T113:
> #0 impala::RuntimeFilterBank::CancelLocked() /home/stakiar/Impala/be/src/runtime/runtime-filter-bank.cc:440 (unifiedbetests+0x2da3475)
> #1 impala::RuntimeFilterBank::Cancel() /home/stakiar/Impala/be/src/runtime/runtime-filter-bank.cc:431 (unifiedbetests+0x2da330a)
> #2 impala::QueryState::Cancel() /home/stakiar/Impala/be/src/runtime/query-state.cc:768 (unifiedbetests+0x2d7f54f)
> #3 impala::ControlService::CancelQueryFInstances(impala::CancelQueryFInstancesRequestPB const*, impala::CancelQueryFInstancesResponsePB*, kudu::rpc::RpcContext*) /home/stakiar/Impala/be/src/service/co
> ntrol-service.cc:234 (unifiedbetests+0x2eee30b)
> #4 operator() /home/stakiar/Impala/be/generated-sources/gen-cpp/control_service.service.cc:110 (unifiedbetests+0x2fa3974)
> #5 std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), impala::ControlServiceIf::ControlServiceIf(scoped_refptr<kudu::MetricEntity> cons
> t&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_5>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) /home/stakiar/Impala/toolc
> hain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039 (unifiedbetests+0x2fa375e)
> #6 std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcCo
> ntext*) const /home/stakiar/Impala/toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439 (unifiedbetests+0x36940e8)
> #7 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /home/stakiar/Impala/be/src/kudu/rpc/service_if.cc:139 (unifiedbetests+0x3693a7e)
> #8 impala::ImpalaServicePool::RunThread() /home/stakiar/Impala/be/src/rpc/impala-service-pool.cc:272 (unifiedbetests+0x36ab83f)
> #9 boost::_mfi::mf0<void, impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49 (unifiedbete
> sts+0x36b0856)
> #10 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<vo
> id, impala::ImpalaServicePool>&, boost::_bi::list0&, int) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:259 (unifiedbetests+0x36b07aa)
> #11 boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >::operator()() /home/stakiar/Impala/toolchain/boost-1
> .61.0-p2/include/boost/bind/bind.hpp:1222 (unifiedbetests+0x36b0733)
> #12 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >,
> void>::invoke(boost::detail::function::function_buffer&) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/function/function_template.hpp:159 (unifiedbetests+0x36b0529)
> #13 boost::function0<void>::operator()() const /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/function/function_template.hpp:770 (unifiedbetests+0x2bc2061)
> #14 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /home/stakiar/Impa
> la/be/src/util/thread.cc:360 (unifiedbetests+0x3533c86)
> #15 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<
> impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::Prom
> iseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseM
> ode)0>*), boost::_bi::list0&, int) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:531 (unifiedbetests+0x353be7c)
> #16 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::lis
> t5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impal
> a::PromiseMode)0>*> > >::operator()() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222 (unifiedbetests+0x353bd93)
> #17 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::Promis
> eMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<i
> mpala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/thread/detail/thread.hpp:116 (unifiedbetests+0x353ba80)
> #18 thread_proxy thread.cpp:? (unifiedbetests+0x37c7869)
> Location is heap block of size 200 at 0x7b340005c520 allocated by thread T113:
> #0 operator new(unsigned long) /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:44 (discriminator 2) (unifiedbetests+0x1c9c0b3)
> #1 impala::QueryState::InitFilterBank() /home/stakiar/Impala/be/src/runtime/query-state.cc:341 (unifiedbetests+0x2d7d501)
> #2 impala::QueryState::Init(impala::ExecQueryFInstancesRequestPB const*, impala::TExecPlanFragmentInfo const&) /home/stakiar/Impala/be/src/runtime/query-state.cc:232 (unifiedbetests+0x2d7c77c)
> #3 impala::QueryExecMgr::StartQuery(impala::ExecQueryFInstancesRequestPB const*, impala::TQueryCtx const&, impala::TExecPlanFragmentInfo const&) /home/stakiar/Impala/be/src/runtime/query-exec-mgr.cc:6
> 0 (unifiedbetests+0x2df6817)
> #4 impala::ControlService::ExecQueryFInstances(impala::ExecQueryFInstancesRequestPB const*, impala::ExecQueryFInstancesResponsePB*, kudu::rpc::RpcContext*) /home/stakiar/Impala/be/src/service/control-
> service.cc:157 (unifiedbetests+0x2eed27e)
> #5 operator() /home/stakiar/Impala/be/generated-sources/gen-cpp/control_service.service.cc:70 (unifiedbetests+0x2fa2474)
> #6 std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), impala::ControlServiceIf::ControlServiceIf(scoped_refptr<kudu::MetricEntity> cons
> t&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) /home/stakiar/Impala/toolc
> hain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039 (unifiedbetests+0x2fa225e)
> #7 std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcCo
> ntext*) const /home/stakiar/Impala/toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439 (unifiedbetests+0x36940e8)
> #8 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /home/stakiar/Impala/be/src/kudu/rpc/service_if.cc:139 (unifiedbetests+0x3693a7e)
> #9 impala::ImpalaServicePool::RunThread() /home/stakiar/Impala/be/src/rpc/impala-service-pool.cc:272 (unifiedbetests+0x36ab83f)
> #10 boost::_mfi::mf0<void, impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49 (unifiedbet
> ests+0x36b0856)
> #11 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<v$
> id, impala::ImpalaServicePool>&, boost::_bi::list0&, int) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:259 (unifiedbetests+0x36b07aa)
> #12 boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >::operator()() /home/stakiar/Impala/toolchain/boost-1
> .61.0-p2/include/boost/bind/bind.hpp:1222 (unifiedbetests+0x36b0733)
> #13 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >,
> void>::invoke(boost::detail::function::function_buffer&) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/function/function_template.hpp:159 (unifiedbetests+0x36b0529)
> #14 boost::function0<void>::operator()() const /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/function/function_template.hpp:770 (unifiedbetests+0x2bc2061)
> #15 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /home/stakiar/Impa
> la/be/src/util/thread.cc:360 (unifiedbetests+0x3533c86)
> #16 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<
> impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::Prom
> iseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseM
> ode)0>*), boost::_bi::list0&, int) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:531 (unifiedbetests+0x353be7c)
> #17 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::lis
> t5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impal
> a::PromiseMode)0>*> > >::operator()() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222 (unifiedbetests+0x353bd93)
> #18 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::Promis
> eMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<i
> mpala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/thread/detail/thread.hpp:116 (unifiedbetests+0x353ba80)
> #19 thread_proxy thread.cpp:? (unifiedbetests+0x37c7869)
> Thread T370 (tid=19643, running) created by thread T113 at:
> #0 pthread_create /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 (unifiedbetests+0x1c33a4b)
> #1 boost::thread::start_thread_noexcept() ??:? (unifiedbetests+0x37c6bf9)
> #2 thread<void (*)(const std::basic_string<char> &, const std::basic_string<char> &, boost::function<void ()>, const impala::ThreadDebugInfo *, impala::Promise<long, impala::PromiseMode::SINGLE_PRODUC
> ER> *), std::basic_string<char>, std::basic_string<char>, boost::function<void ()>, impala::ThreadDebugInfo *, impala::Promise<long, impala::PromiseMode::SINGLE_PRODUCER> *> /home/stakiar/Impala/toolchain
> /boost-1.61.0-p2/include/boost/thread/detail/thread.hpp:420 (unifiedbetests+0x353645b)
> #3 impala::Thread::StartThread(std::string const&, std::string const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /home/stakiar/Impa
> la/be/src/util/thread.cc:317 (unifiedbetests+0x3533625)
> #4 impala::Status impala::Thread::Create<void (impala::QueryExecMgr::*)(impala::QueryState*), impala::QueryExecMgr*, impala::QueryState*>(std::string const&, std::string const&, void (impala::QueryExe
> cMgr::* const&)(impala::QueryState*), impala::QueryExecMgr* const&, impala::QueryState* const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /home/stakiar/Impala/be/src/ut
> il/thread.h:89 (unifiedbetests+0x2df7aa7)
> #5 impala::QueryExecMgr::StartQuery(impala::ExecQueryFInstancesRequestPB const*, impala::TQueryCtx const&, impala::TExecPlanFragmentInfo const&) /home/stakiar/Impala/be/src/runtime/query-exec-mgr.cc:6
> 9 (unifiedbetests+0x2df6992)
> #6 impala::ControlService::ExecQueryFInstances(impala::ExecQueryFInstancesRequestPB const*, impala::ExecQueryFInstancesResponsePB*, kudu::rpc::RpcContext*) /home/stakiar/Impala/be/src/service/control-
> service.cc:157 (unifiedbetests+0x2eed27e)
> #7 operator() /home/stakiar/Impala/be/generated-sources/gen-cpp/control_service.service.cc:70 (unifiedbetests+0x2fa2474)
> #8 std::_Function_handler<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*), impala::ControlServiceIf::ControlServiceIf(scoped_refptr<kudu::MetricEntity> cons
> t&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1>::_M_invoke(std::_Any_data const&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) /home/stakiar/Impala/toolc
> hain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2039 (unifiedbetests+0x2fa225e)
> #9 std::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcC$
> ntext*) const /home/stakiar/Impala/toolchain/gcc-4.9.2/lib/gcc/x86_64-unknown-linux-gnu/4.9.2/../../../../include/c++/4.9.2/functional:2439 (unifiedbetests+0x36940e8)
> #10 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /home/stakiar/Impala/be/src/kudu/rpc/service_if.cc:139 (unifiedbetests+0x3693a7e)
> #11 impala::ImpalaServicePool::RunThread() /home/stakiar/Impala/be/src/rpc/impala-service-pool.cc:272 (unifiedbetests+0x36ab83f)
> #12 boost::_mfi::mf0<void, impala::ImpalaServicePool>::operator()(impala::ImpalaServicePool*) const /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/mem_fn_template.hpp:49 (unifiedbet
> ests+0x36b0856)
> #13 void boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> >::operator()<boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<vo
> id, impala::ImpalaServicePool>&, boost::_bi::list0&, int) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:259 (unifiedbetests+0x36b07aa)
> #14 boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >::operator()() /home/stakiar/Impala/toolchain/boost-1
> .61.0-p2/include/boost/bind/bind.hpp:1222 (unifiedbetests+0x36b0733)
> #15 boost::detail::function::void_function_obj_invoker0<boost::_bi::bind_t<void, boost::_mfi::mf0<void, impala::ImpalaServicePool>, boost::_bi::list1<boost::_bi::value<impala::ImpalaServicePool*> > >,
> void>::invoke(boost::detail::function::function_buffer&) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/function/function_template.hpp:159 (unifiedbetests+0x36b0529)
> #16 boost::function0<void>::operator()() const /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/function/function_template.hpp:770 (unifiedbetests+0x2bc2061)
> #17 impala::Thread::SuperviseThread(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*) /home/stakiar/Impa
> la/be/src/util/thread.cc:360 (unifiedbetests+0x3533c86)
> #18 void boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<
> impala::Promise<long, (impala::PromiseMode)0>*> >::operator()<void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::Prom
> iseMode)0>*), boost::_bi::list0>(boost::_bi::type<void>, void (*&)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseM
> ode)0>*), boost::_bi::list0&, int) /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:531 (unifiedbetests+0x353be7c)
> #19 boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::PromiseMode)0>*), boost::_bi::lis
> t5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<impala::Promise<long, (impal
> a::PromiseMode)0>*> > >::operator()() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/bind/bind.hpp:1222 (unifiedbetests+0x353bd93)
> #20 boost::detail::thread_data<boost::_bi::bind_t<void, void (*)(std::string const&, std::string const&, boost::function<void ()>, impala::ThreadDebugInfo const*, impala::Promise<long, (impala::Promis
> eMode)0>*), boost::_bi::list5<boost::_bi::value<std::string>, boost::_bi::value<std::string>, boost::_bi::value<boost::function<void ()> >, boost::_bi::value<impala::ThreadDebugInfo*>, boost::_bi::value<i
> mpala::Promise<long, (impala::PromiseMode)0>*> > > >::run() /home/stakiar/Impala/toolchain/boost-1.61.0-p2/include/boost/thread/detail/thread.hpp:116 (unifiedbetests+0x353ba80)
> #21 thread_proxy thread.cpp:? (unifiedbetests+0x37c7869) Thread T113 (tid=30180, running) created by main thread at:
> #0 pthread_create /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 (unifiedbetests+0x1c33a4b)
> #1 boost::thread::start_thread_noexcept() ??:? (unifiedbetests+0x37c6bf9)
> #2 thread<void (*)(const std::basic_string<char> &, const std::basic_string<char> &, boost::function<void ()>, const impala::ThreadDebugInfo *, impala::Promise<long, impala::PromiseMode::SINGLE_PRODUC
> ER> *), std::basic_string<char>, std::basic_string<char>, boost::function<void ()>, impala::ThreadDebugInfo *, impala::Promise<long, impala::PromiseMode::SINGLE_PRODUCER> *> /home/stakiar/Impala/toolchain
> /boost-1.61.0-p2/include/boost/thread/detail/thread.hpp:420 (unifiedbetests+0x353645b)
> #3 impala::Thread::StartThread(std::string const&, std::string const&, boost::function<void ()> const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /home/stakiar/Impa
> la/be/src/util/thread.cc:317 (unifiedbetests+0x3533625)
> #4 impala::Status impala::Thread::Create<void (impala::ImpalaServicePool::*)(), impala::ImpalaServicePool*>(std::string const&, std::string const&, void (impala::ImpalaServicePool::* const&)(), impala
> ::ImpalaServicePool* const&, std::unique_ptr<impala::Thread, std::default_delete<impala::Thread> >*, bool) /home/stakiar/Impala/be/src/util/thread.h:81 (unifiedbetests+0x36ad877)
> #5 impala::ImpalaServicePool::Init(int) /home/stakiar/Impala/be/src/rpc/impala-service-pool.cc:92 (unifiedbetests+0x36ab585)
> #6 impala::RpcMgr::RegisterService(int, int, kudu::rpc::GeneratedServiceIf*, impala::MemTracker*) /home/stakiar/Impala/be/src/rpc/rpc-mgr.cc:163 (unifiedbetests+0x3694c54)
> #7 impala::ControlService::Init() /home/stakiar/Impala/be/src/service/control-service.cc:80 (unifiedbetests+0x2eeca70)
> #8 impala::ExecEnv::Init() /home/stakiar/Impala/be/src/runtime/exec-env.cc:387 (unifiedbetests+0x2d07404)
> #9 impala::InProcessImpalaServer::StartWithClientServers(int, int, int) /home/stakiar/Impala/be/src/testutil/in-process-servers.cc:81 (unifiedbetests+0x30270f0)
> #10 impala::InProcessImpalaServer::StartWithEphemeralPorts(std::string const&, int, impala::InProcessImpalaServer**) /home/stakiar/Impala/be/src/testutil/in-process-servers.cc:60 (unifiedbetests+0x302
> 7079)
> #11 impala::ExprTest::SetUpTestCase() /home/stakiar/Impala/be/src/exprs/expr-test.cc:231 (unifiedbetests+0x1e17b78)
> #12 testing::TestCase::Run() ??:? (unifiedbetests+0x49a5832)
> #13 __libc_start_main ??:? (libc.so.6+0x2082f)SUMMARY: ThreadSanitizer: data race /home/stakiar/Impala/be/src/runtime/runtime-filter-bank.cc:435 in impala::RuntimeFilterBank::CancelLocked() {code}
> I investigated the issue a bit, and it looks like there is a (seemingly harmless) data race in the cancellation code path of RuntimeFilterBank that can be triggered if no filters are actually registered.
> The cancellation code in RuntimeFilterBank is as follows:
> {code}
> vector<unique_lock<SpinLock>> RuntimeFilterBank::LockAllFilters() {
> vector<unique_lock<SpinLock>> locks;
> for (auto& entry : filters_) locks.emplace_back(entry.second->lock);
> return locks;
> }
> void RuntimeFilterBank::Cancel() {
> auto all_locks = LockAllFilters();
> CancelLocked();
> }
> void RuntimeFilterBank::CancelLocked() {
> if (cancelled_) return;
> // Cancel all filters that a thread might be waiting on.
> for (auto& entry : filters_) {
> ¦ if (entry.second->consumed_filter != nullptr) entry.second->consumed_filter->Cancel();
> }
> cancelled_ = true;
> }
> {code}
> The issue is that if {{filters_}} is empty, then no locks get acquired, which causes a race on {{cancelled_}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)