You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/09/24 07:16:00 UTC
[jira] [Commented] (IMPALA-10170) Data race on
Webserver::UrlHandler::is_on_nav_bar_
[ https://issues.apache.org/jira/browse/IMPALA-10170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17201322#comment-17201322 ]
ASF subversion and git services commented on IMPALA-10170:
----------------------------------------------------------
Commit d6442e0d8afb9fc4f5070583f1ce2ee143b7f13b in impala's branch refs/heads/master from Sahil Takiar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=d6442e0 ]
IMPALA-10170: Data race on Webserver::UrlHandler::is_on_nav_bar_
This data race can be reproduced by
TestCompactCatalogUpdates.test_restart_catalogd, although it does not
seem to always occur. The data race was originally reported in a Jenkins
job, but I could not reproduce it locally.
The fix is to acquire a read lock while reading some UrlHandler objects.
I cleaned up some of the other involved variables and made them const.
These variables are set during construction time, and never modified
afterwards.
Testing:
* Ran be and custom cluster TSAN tests
Change-Id: I6923af4754e3fe72b8b04c5303a1e7a79da7613a
Reviewed-on: http://gerrit.cloudera.org:8080/16459
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Data race on Webserver::UrlHandler::is_on_nav_bar_
> --------------------------------------------------
>
> Key: IMPALA-10170
> URL: https://issues.apache.org/jira/browse/IMPALA-10170
> Project: IMPALA
> Issue Type: Sub-task
> Components: Backend
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
>
> {code}
> WARNING: ThreadSanitizer: data race (pid=31102)
> Read of size 1 at 0x7b2c0006e3b0 by thread T42:
> #0 impala::Webserver::UrlHandler::is_on_nav_bar() const /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.h:152:41 (impalad+0x256ff39)
> #1 impala::Webserver::GetCommonJson(rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, rapidjson::CrtAllocator>*, sq_connection const*, kudu::WebCallbackRegistry::WebRequest const&) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.cc:527:24 (impalad+0x256be13)
> #2 impala::Webserver::RenderUrlWithTemplate(sq_connection const*, kudu::WebCallbackRegistry::WebRequest const&, impala::Webserver::UrlHandler const&, std::__cxx11::basic_stringstream<char, std::char_traits<char>, std::allocator<char> >*, impala::ContentType*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.cc:816:3 (impalad+0x256e882)
> #3 impala::Webserver::BeginRequestCallback(sq_connection*, sq_request_info*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.cc:714:5 (impalad+0x256cfbb)
> #4 impala::Webserver::BeginRequestCallbackStatic(sq_connection*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.cc:556:20 (impalad+0x256ba98)
> #5 handle_request <null> (impalad+0x2582d59)
> Previous write of size 2 at 0x7b2c0006e3b0 by main thread:
> #0 impala::Webserver::UrlHandler::UrlHandler(impala::Webserver::UrlHandler&&) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.h:141:9 (impalad+0x2570dbc)
> #1 std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler, true>(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler>&&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_pair.h:362:4 (impalad+0x25738b3)
> #2 void __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >::construct<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler> >(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>*, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler>&&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/ext/new_allocator.h:136:23 (impalad+0x2573848)
> #3 void std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > > >::construct<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler> >(std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >&, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>*, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler>&&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/alloc_traits.h:475:8 (impalad+0x25737f1)
> #4 void std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >::_M_construct_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler> >(std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >*, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler>&&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_tree.h:626:8 (impalad+0x257369b)
> #5 std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >* std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >::_M_create_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler> >(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler>&&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_tree.h:643:4 (impalad+0x2573309)
> #6 std::pair<std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >, bool> std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >::_M_emplace_unique<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler> >(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler>&&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_tree.h:2351:19 (impalad+0x25731be)
> #7 _ZNSt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN6impala9Webserver10UrlHandlerESt4lessIS5_ESaISt4pairIKS5_S8_EEE6insertISB_IS5_S8_EEENSt9enable_ifIXsr16is_constructibleISD_T_EE5valueESB_ISt17_Rb_tree_iteratorISD_EbEE4typeEOSJ_ /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_map.h:809:16 (impalad+0x25704d8)
> #8 impala::Webserver::RegisterUrlCallback(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void (kudu::WebCallbackRegistry::WebRequest const&, rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, rapidjson::CrtAllocator>*)> const&, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.cc:868:17 (impalad+0x256f3f8)
> #9 impala::RegisterLogLevelCallbacks(impala::Webserver*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/logging-support.cc:320:14 (impalad+0x24d73bd)
> #10 impala::CatalogServer::RegisterWebpages(impala::Webserver*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/catalog/catalog-server.cc:310:3 (impalad+0x1ac10a3)
> #11 CatalogdMain(int, char**) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/catalog/catalogd-main.cc:98:18 (impalad+0x1ab3f89)
> #12 main /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/daemon-main.cc:41:12 (impalad+0x1ab2d9d)
> Location is heap block of size 168 at 0x7b2c0006e370 allocated by main thread:
> #0 operator new(unsigned long) /mnt/source/llvm/llvm-5.0.1.src-p3/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cc:44 (impalad+0x1ab04d6)
> #1 __gnu_cxx::new_allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >::allocate(unsigned long, void const*) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/ext/new_allocator.h:111:27 (impalad+0x257376e)
> #2 std::allocator_traits<std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > > >::allocate(std::allocator<std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >&, unsigned long) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/alloc_traits.h:436:20 (impalad+0x2573712)
> #3 std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >::_M_get_node() /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_tree.h:588:16 (impalad+0x2573626)
> #4 std::_Rb_tree_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >* std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >::_M_create_node<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler> >(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler>&&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_tree.h:642:23 (impalad+0x25732f0)
> #5 std::pair<std::_Rb_tree_iterator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >, bool> std::_Rb_tree<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler>, std::_Select1st<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, impala::Webserver::UrlHandler> > >::_M_emplace_unique<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler> >(std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, impala::Webserver::UrlHandler>&&) /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_tree.h:2351:19 (impalad+0x25731be)
> #6 _ZNSt3mapINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN6impala9Webserver10UrlHandlerESt4lessIS5_ESaISt4pairIKS5_S8_EEE6insertISB_IS5_S8_EEENSt9enable_ifIXsr16is_constructibleISD_T_EE5valueESB_ISt17_Rb_tree_iteratorISD_EbEE4typeEOSJ_ /data/jenkins/workspace/impala-private-parameterized/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_map.h:809:16 (impalad+0x25704d8)
> #7 impala::Webserver::RegisterUrlCallback(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, boost::function<void (kudu::WebCallbackRegistry::WebRequest const&, rapidjson::GenericDocument<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>, rapidjson::CrtAllocator>*)> const&, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.cc:868:17 (impalad+0x256f3f8)
> #8 impala::RegisterLogLevelCallbacks(impala::Webserver*, bool) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/logging-support.cc:320:14 (impalad+0x24d73bd)
> #9 impala::CatalogServer::RegisterWebpages(impala::Webserver*) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/catalog/catalog-server.cc:310:3 (impalad+0x1ac10a3)
> #10 CatalogdMain(int, char**) /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/catalog/catalogd-main.cc:98:18 (impalad+0x1ab3f89)
> #11 main /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/service/daemon-main.cc:41:12 (impalad+0x1ab2d9d)
> Thread T42 'sq_worker' (tid=31159, running) created by thread T40 at:
> #0 pthread_create /mnt/source/llvm/llvm-5.0.1.src-p3/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors.cc:889 (impalad+0x1a24026)
> #1 sq_start_thread <null> (impalad+0x2579a24)
> SUMMARY: ThreadSanitizer: data race /data/jenkins/workspace/impala-private-parameterized/repos/Impala/be/src/util/webserver.h:152:41 in impala::Webserver::UrlHandler::is_on_nav_bar() const
> ==================
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org