You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kvrocks.apache.org by GitBox <gi...@apache.org> on 2022/06/30 07:16:47 UTC
[GitHub] [incubator-kvrocks] PragmaTwice opened a new issue, #699: BUG: Use after free in CommandBPop reported by ASan
PragmaTwice opened a new issue, #699:
URL: https://github.com/apache/incubator-kvrocks/issues/699
### Search before asking
- [X] I had searched in the [issues](https://github.com/apache/incubator-kvrocks/issues) and found no similar issues.
### Version
unstable
### Minimal reproduce step
```
./build.sh build -DENABLE_ASAN=ON
cd tests/tcl
./runtest --dont-clean
grep Sanitizer tests/tmp -r
```
This bug is not 100% reproducible and may require multiple test executions before it appears.
### What did you expect to see?
No ASan report.
### What did you see instead?
```
=================================================================
==1872112==ERROR: AddressSanitizer: heap-use-after-free on address 0x608000014070 at pc 0x55d1a126595a bp 0x7fa436fee820 sp 0x7fa436fee810
READ of size 8 at 0x608000014070 thread T98
#0 0x55d1a1265959 in Redis::CommandBPop::TryPopFromList() /home/twice/incubator-kvrocks/src/redis_cmd.cc:1568
#1 0x55d1a1266851 in Redis::CommandBPop::WriteCB(bufferevent*, void*) /home/twice/incubator-kvrocks/src/redis_cmd.cc:1590
#2 0x55d1a190f519 in bufferevent_run_deferred_callbacks_unlocked /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/bufferevent.c:215
#3 0x55d1a19169ac in event_process_active_single_queue /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/event.c:1726
#4 0x55d1a1917406 in event_process_active /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/event.c:1789
#5 0x55d1a1917406 in event_base_loop /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/event.c:2012
#6 0x55d1a14ffe9f in Worker::Run(std::thread::id) /home/twice/incubator-kvrocks/src/worker.cc:234
#7 0x55d1a150017d in operator() /home/twice/incubator-kvrocks/src/worker.cc:435
#8 0x55d1a150017d in __invoke_impl<void, WorkerThread::Start()::<lambda()> > /usr/include/c++/11.2.0/bits/invoke.h:61
#9 0x55d1a150017d in __invoke<WorkerThread::Start()::<lambda()> > /usr/include/c++/11.2.0/bits/invoke.h:96
#10 0x55d1a150017d in _M_invoke<0> /usr/include/c++/11.2.0/bits/std_thread.h:253
#11 0x55d1a150017d in operator() /usr/include/c++/11.2.0/bits/std_thread.h:260
#12 0x55d1a150017d in _M_run /usr/include/c++/11.2.0/bits/std_thread.h:211
#13 0x55d1a19fa4a3 in execute_native_thread_routine (/home/twice/incubator-kvrocks/tests/tcl/redis-server+0xacd4a3)
#14 0x7fa440e6b5c1 in start_thread (/usr/lib/libc.so.6+0x8d5c1)
#15 0x7fa440ef0583 in __clone (/usr/lib/libc.so.6+0x112583)
0x608000014070 is located 80 bytes inside of 96-byte region [0x608000014020,0x608000014080)
freed by thread T98 here:
#0 0x7fa4411a48d9 in operator delete(void*, unsigned long) /usr/src/debug/gcc/libsanitizer/asan/asan_new_delete.cpp:172
#1 0x55d1a1424770 in std::default_delete<Redis::Commander>::operator()(Redis::Commander*) const /usr/include/c++/11.2.0/bits/unique_ptr.h:85
#2 0x55d1a1424770 in std::__uniq_ptr_impl<Redis::Commander, std::default_delete<Redis::Commander> >::reset(Redis::Commander*) /usr/include/c++/11.2.0/bits/unique_ptr.h:182
#3 0x55d1a1424770 in std::__uniq_ptr_impl<Redis::Commander, std::default_delete<Redis::Commander> >::operator=(std::__uniq_ptr_impl<Redis::Commander, std::default_delete<Redis::Commander> >&&) /usr/include/c++/11.2.0/bits/unique_ptr.h:167
#4 0x55d1a1424770 in std::__uniq_ptr_data<Redis::Commander, std::default_delete<Redis::Commander>, true, true>::operator=(std::__uniq_ptr_data<Redis::Commander, std::default_delete<Redis::Commander>, true, true>&&) /usr/include/c++/11.2.0/bits/unique_ptr.h:212
#5 0x55d1a1424770 in std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >::operator=(std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >&&) /usr/include/c++/11.2.0/bits/unique_ptr.h:371
#6 0x55d1a1424770 in Server::LookupAndCreateCommand(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >*) /home/twice/incubator-kvrocks/src/server.cc:1340
#7 0x55d1a12c65ad in Redis::Connection::ExecuteCommands(std::vector<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::allocator<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > const&) /home/twice/incubator-kvrocks/src/redis_connection.cc:330
#8 0x55d1a12cef3f in Redis::Connection::OnRead(bufferevent*, void*) /home/twice/incubator-kvrocks/src/redis_connection.cc:89
#9 0x55d1a190f4ad in bufferevent_run_deferred_callbacks_unlocked /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/bufferevent.c:208
previously allocated by thread T98 here:
#0 0x7fa4411a3811 in operator new(unsigned long) /usr/src/debug/gcc/libsanitizer/asan/asan_new_delete.cpp:99
#1 0x55d1a11c3c56 in Redis::redisCommandTable::{lambda()#81}::operator()() const /home/twice/incubator-kvrocks/src/redis_cmd.cc:4807
#2 0x55d1a11c3c56 in std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> > std::__invoke_impl<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >, Redis::redisCommandTable::{lambda()#81}&>(std::__invoke_other, Redis::redisCommandTable::{lambda()#81}&) /usr/include/c++/11.2.0/bits/invoke.h:61
#3 0x55d1a11c3c56 in std::enable_if<is_invocable_r_v<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >, Redis::redisCommandTable::{lambda()#81}&>, std::enable_if>::type std::__invoke_r<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >, Redis::redisCommandTable::{lambda()#81}&>(std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >&&, (Redis::redisCommandTable::{lambda()#81}&)...) /usr/include/c++/11.2.0/bits/invoke.h:116
#4 0x55d1a11c3c56 in std::_Function_handler<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> > (), Redis::redisCommandTable::{lambda()#81}>::_M_invoke(std::_Any_data const&) /usr/include/c++/11.2.0/bits/std_function.h:292
#5 0x55d1a142439e in std::function<std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> > ()>::operator()() const /usr/include/c++/11.2.0/bits/std_function.h:560
#6 0x55d1a142439e in Server::LookupAndCreateCommand(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<Redis::Commander, std::default_delete<Redis::Commander> >*) /home/twice/incubator-kvrocks/src/server.cc:1340
#7 0x55d1a12c65ad in Redis::Connection::ExecuteCommands(std::vector<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, std::allocator<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > const&) /home/twice/incubator-kvrocks/src/redis_connection.cc:330
#8 0x55d1a12cef3f in Redis::Connection::OnRead(bufferevent*, void*) /home/twice/incubator-kvrocks/src/redis_connection.cc:89
#9 0x55d1a190f4ad in bufferevent_run_deferred_callbacks_unlocked /home/twice/incubator-kvrocks/sanbuild/_deps/libevent-src/bufferevent.c:208
Thread T98 created by T0 here:
#0 0x7fa441143eb7 in __interceptor_pthread_create /usr/src/debug/gcc/libsanitizer/asan/asan_interceptors.cpp:216
#1 0x55d1a19fa78a in std::thread::_M_start_thread(std::unique_ptr<std::thread::_State, std::default_delete<std::thread::_State> >, void (*)()) (/home/twice/incubator-kvrocks/tests/tcl/redis-server+0xacd78a)
#2 0x55d1a1443c6b in Server::Start() /home/twice/incubator-kvrocks/src/server.cc:125
#3 0x55d1a1086cf5 in main /home/twice/incubator-kvrocks/src/main.cc:354
#4 0x7fa440e0b30f in __libc_start_call_main (/usr/lib/libc.so.6+0x2d30f)
SUMMARY: AddressSanitizer: heap-use-after-free /home/twice/incubator-kvrocks/src/redis_cmd.cc:1568 in Redis::CommandBPop::TryPopFromList()
Shadow bytes around the buggy address:
0x0c107fffa7b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c107fffa7c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c107fffa7d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c107fffa7e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c107fffa7f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c107fffa800: fa fa fa fa fd fd fd fd fd fd fd fd fd fd[fd]fd
0x0c107fffa810: fa fa fa fa fd fd fd fd fd fd fd fd fd fd fd fd
0x0c107fffa820: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 fa
0x0c107fffa830: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
0x0c107fffa840: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00
0x0c107fffa850: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==1872112==ABORTING
```
### Anything Else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] PragmaTwice commented on issue #699: BUG: Use after free in CommandBPop reported by ASan
Posted by GitBox <gi...@apache.org>.
PragmaTwice commented on issue #699:
URL: https://github.com/apache/incubator-kvrocks/issues/699#issuecomment-1170869347
I think we may need to change some logic: we may not be able to use a pattern like `unique_ptr<Commander> Connection::current_cmd_`, because that makes the current command get destructured when the next command is read in.
There may be commands, such as `CommandBPop`, that need to have a longer lifetime and not be destructured immediately before the next command is executed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] PragmaTwice commented on issue #699: BUG: Use after free in CommandBPop reported by ASan
Posted by GitBox <gi...@apache.org>.
PragmaTwice commented on issue #699:
URL: https://github.com/apache/incubator-kvrocks/issues/699#issuecomment-1170861979
I think I found the root cause:
The the current `CommandBPop` is deallocated while `CommandBPop::WriteCB` is called by bufferevent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] git-hulk commented on issue #699: BUG: Use after free in CommandBPop reported by ASan
Posted by GitBox <gi...@apache.org>.
git-hulk commented on issue #699:
URL: https://github.com/apache/incubator-kvrocks/issues/699#issuecomment-1170893461
Yes, maybe we can use shared_ptr to workaround this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] git-hulk closed issue #699: BUG: Use after free in CommandBPop reported by ASan
Posted by GitBox <gi...@apache.org>.
git-hulk closed issue #699: BUG: Use after free in CommandBPop reported by ASan
URL: https://github.com/apache/incubator-kvrocks/issues/699
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-kvrocks] git-hulk commented on issue #699: BUG: Use after free in CommandBPop reported by ASan
Posted by GitBox <gi...@apache.org>.
git-hulk commented on issue #699:
URL: https://github.com/apache/incubator-kvrocks/issues/699#issuecomment-1188907048
@PragmaTwice I fixed this issue in #714, can take a look if you have free time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@kvrocks.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org