You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "ASF subversion and git services (Jira)" <ji...@apache.org> on 2020/06/22 22:11:00 UTC
[jira] [Commented] (IMPALA-9878) Use-after-free in
tmp-file-mgr-test.cc
[ https://issues.apache.org/jira/browse/IMPALA-9878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17142445#comment-17142445 ]
ASF subversion and git services commented on IMPALA-9878:
---------------------------------------------------------
Commit 7b1cfacbc6c4c709947cb91517baa9ec364afee1 in impala's branch refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=7b1cfac ]
IMPALA-9878: Fix use-after-free in TmpFileMgrTest's TestAllocation
ASAN found a use-after-free for the in this code:
file_group.Close(); <--- free underlying storage for 'file'
EXPECT_FALSE(boost::filesystem::exists(file->path())); <-- use 'file'
This switches it to a copy of file->path().
Testing:
- Ran tmp-file-mgr-test under ASAN
Change-Id: Idd5cbae70c287c78db8d1c560d8c777d6bed5b56
Reviewed-on: http://gerrit.cloudera.org:8080/16099
Reviewed-by: Tim Armstrong <ta...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
> Use-after-free in tmp-file-mgr-test.cc
> ---------------------------------------
>
> Key: IMPALA-9878
> URL: https://issues.apache.org/jira/browse/IMPALA-9878
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 4.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Blocker
> Labels: broken-build
>
> The ASAN build detected a use-after-free from TmpFileMgrTest's TestFileAllocation:
>
> {noformat}
> ==14993==ERROR: AddressSanitizer: heap-use-after-free on address 0x608000079aa0 at pc 0x000001d2347e bp 0x7fff686cc130 sp 0x7fff686cb8e0
> READ of size 90 at 0x608000079aa0 thread T0
> #0 0x1d2347d in __interceptor_memcpy.part.40 /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:738
> #1 0x1e101b3 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/basic_string.tcc:225:6
> #2 0x7fb4c8b1c72e in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct_aux<char*>(char*, char*, std::__false_type) /mnt/source/gcc/build-7.5.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:236
> #3 0x7fb4c8b1c72e in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*) /mnt/source/gcc/build-7.5.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:255
> #4 0x7fb4c8b1c72e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /mnt/source/gcc/build-7.5.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:440
> #5 0x236cfce in impala::TmpFileMgrTest_TestFileAllocation_Test::TestBody() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr-test.cc:281:3
> #6 0x61b5ac9 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x61b5ac9)
> #7 0x61aeef9 in testing::Test::Run() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x61aeef9)
> #8 0x61aefdb in testing::TestInfo::Run() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x61aefdb)
> #9 0x61af114 in testing::TestCase::Run() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x61af114)
> #10 0x61af7bf in testing::internal::UnitTestImpl::RunAllTests() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x61af7bf)
> #11 0x61af8f6 in testing::UnitTest::Run() (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x61af8f6)
> #12 0x1dfc876 in main /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/service/unified-betest-main.cc:48:10
> #13 0x7fb4c8140c04 in __libc_start_main (/lib64/libc.so.6+0x21c04)
> #14 0x1d05506 in _start (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x1d05506)
> 0x608000079aa0 is located 0 bytes inside of 91-byte region [0x608000079aa0,0x608000079afb)
> freed by thread T0 here:
> #0 0x1df9040 in operator delete(void*) /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/asan/asan_new_delete.cc:137
> #1 0x2398575 in std::default_delete<impala::TmpFile>::operator()(impala::TmpFile*) const /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/unique_ptr.h:78:2
> #2 0x238f806 in std::unique_ptr<impala::TmpFile, std::default_delete<impala::TmpFile> >::~unique_ptr() /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/unique_ptr.h:263:4
> #3 0x3c397af in void std::_Destroy_aux<false>::__destroy<std::unique_ptr<impala::TmpFile, std::default_delete<impala::TmpFile> >*>(std::unique_ptr<impala::TmpFile, std::default_delete<impala::TmpFile> >*, std::unique_ptr<impala::TmpFile, std::default_delete<impala::TmpFile> >*) /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_construct.h:108:6
> #4 0x3c3acd8 in std::vector<std::unique_ptr<impala::TmpFile, std::default_delete<impala::TmpFile> >, std::allocator<std::unique_ptr<impala::TmpFile, std::default_delete<impala::TmpFile> > > >::_M_erase_at_end(std::unique_ptr<impala::TmpFile, std::default_delete<impala::TmpFile> >*) /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/stl_vector.h:1518:2
> #5 0x3c285c3 in impala::TmpFileGroup::Close() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:439:14
> #6 0x236cfa5 in impala::TmpFileMgrTest_TestFileAllocation_Test::TestBody() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr-test.cc:280:14
> #7 0x61b5ac9 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x61b5ac9)
> previously allocated by thread T0 here:
> #0 0x1df82d0 in operator new(unsigned long) /mnt/source/llvm/llvm-5.0.1.src-p2/projects/compiler-rt/lib/asan/asan_new_delete.cc:92
> #1 0x1e1016e in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) /data/jenkins/workspace/impala-asf-master-core-asan/Impala-Toolchain/toolchain-packages-gcc7.5.0/gcc-7.5.0/lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/basic_string.tcc:219:14
> #2 0x7fb4c8b1c72e in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct_aux<char*>(char*, char*, std::__false_type) /mnt/source/gcc/build-7.5.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:236
> #3 0x7fb4c8b1c72e in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*) /mnt/source/gcc/build-7.5.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:255
> #4 0x7fb4c8b1c72e in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /mnt/source/gcc/build-7.5.0/x86_64-pc-linux-gnu/libstdc++-v3/include/bits/basic_string.h:440
> #5 0x3c23a82 in impala::TmpFileMgr::NewFile(impala::TmpFileGroup*, int, std::unique_ptr<impala::TmpFile, std::default_delete<impala::TmpFile> >*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:284:23
> #6 0x3c2756f in impala::TmpFileGroup::CreateFiles() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr.cc:414:20
> #7 0x238e216 in impala::TmpFileMgrTest::CreateFiles(impala::TmpFileGroup*, std::vector<impala::TmpFile*, std::allocator<impala::TmpFile*> >*) /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr-test.cc:142:5
> #8 0x236c6de in impala::TmpFileMgrTest_TestFileAllocation_Test::TestBody() /data/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/src/runtime/tmp-file-mgr-test.cc:260:3
> #9 0x61b5ac9 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) (/data0/jenkins/workspace/impala-asf-master-core-asan/repos/Impala/be/build/debug/service/unifiedbetests+0x61b5ac9){noformat}
> The problem is here:
>
> {noformat}
> // Check that the file is cleaned up correctly. Need to create file first since
> // tmp file is only allocated on writes.
> EXPECT_OK(FileSystemUtil::CreateFile(file->path()));
> file_group.Close();
> EXPECT_FALSE(boost::filesystem::exists(file->path())); <-------{noformat}
> [https://github.com/apache/impala/blob/master/be/src/runtime/tmp-file-mgr-test.cc#L281]
> "file" is a pointer into the the file_group, so when file_group.Close() runs, that gets freed.
> This must be newly detected after the GCC7 change.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org