You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by GitBox <gi...@apache.org> on 2021/01/08 12:47:42 UTC
[GitHub] [incubator-doris] stdpain opened a new issue #5213: BE may probabilistic trigger segmentfault when BE exit
stdpain opened a new issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213
**Describe the bug**
BE may probabilistic trigger segmentfault when BE exit
This bug will not affect the function, but it may increase the difficulty of subsequent troubleshooting such as heap-profile
here is a coredump (master build with debug)
```
Core was generated by `/home/users/stdpain/opt/doris-deploy/be/lib/palo_be'.
Program terminated with signal SIGSEGV, Segmentation fault.
b#0 0x00007ff97bb7d09c in __gnu_cxx::__normal_iterator<doris::TabletManager::tablets_shard*, std::vector<doris::TabletManager::tablets_shard, std::allocator<doris::TabletManager::tablets_shard> > >::__normal_iterator (this=0x7ff904d442b8, __i=<error reading variable>)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/stl_iterator.h:780
780 : _M_current(__i) { }
[Current thread is 1 (LWP 38702)]
warning: File "/ssd1/opt/fenghaoasuch/workspace/doris/workspace/doris-toolchain/gcc730/lib64/libstdc++.so.6.0.24-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
(gdb) bt
#0 0x00007ff97bb7d09c in __gnu_cxx::__normal_iterator<doris::TabletManager::tablets_shard*, std::vector<doris::TabletManager::tablets_shard, std::allocator<doris::TabletManager::tablets_shard> > >::__normal_iterator (this=0x7ff904d442b8, __i=<error reading variable>)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/stl_iterator.h:780
#1 0x00007ff97bb7b3df in std::vector<doris::TabletManager::tablets_shard, std::allocator<doris::TabletManager::tablets_shard> >::begin (this=0x8)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/stl_vector.h:564
#2 0x00007ff97bb6ed37 in doris::TabletManager::find_best_tablet_to_compaction (this=0x0,
compaction_type=doris::CUMULATIVE_COMPACTION, data_dir=0x55fca00,
tablet_submitted_compaction=std::vector of length 0, capacity 0)
at /home/users/stdpain/doris/core/be/src/olap/tablet_manager.cpp:681
#3 0x00007ff97ba76a83 in doris::StorageEngine::_compaction_tasks_generator (this=0x558cc00,
compaction_type=doris::CUMULATIVE_COMPACTION,
data_dirs=std::vector of length 1, capacity 1 = {...})
at /home/users/stdpain/doris/core/be/src/olap/olap_server.cpp:397
#4 0x00007ff97ba764d5 in doris::StorageEngine::_compaction_tasks_producer_callback (this=0x558cc00)
at /home/users/stdpain/doris/core/be/src/olap/olap_server.cpp:337
#5 0x00007ff97ba73d39 in doris::StorageEngine::<lambda()>::operator()(void) const (
__closure=0x6fb8f18) at /home/users/stdpain/doris/core/be/src/olap/olap_server.cpp:78
#6 0x00007ff97ba77ae1 in std::_Function_handler<void(), doris::StorageEngine::start_bg_threads()::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/std_function.h:316
#7 0x00007ff97cf14b7c in std::function<void ()>::operator()() const (this=0x6fb8f18)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/std_function.h:706
#8 0x00007ff97a7163ce in doris::Thread::supervise_thread (arg=0x6fb8f00)
at /home/users/stdpain/doris/core/be/src/util/thread.cpp:386
#9 0x00007ff978cd21c3 in start_thread () from /opt/compiler/gcc-4.8.2/lib64/libpthread.so.0
#10 0x00007ff9782f512d in clone () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
```
Here was be.out when rebuild with ASAN:
```
=================================================================
==54102==ERROR: AddressSanitizer: heap-use-after-free on address 0x6190000cddc8 at pc 0x000001d36929 bp 0x7fcbbb572b70 sp 0x7fcbbb572b68
READ of size 8 at 0x6190000cddc8 thread T233 (compaction_task)
#0 0x1d36928 in std::_Rb_tree<doris::DataDir*, std::pair<doris::DataDir* const, std::vector<long, std::allocator<long> > >, std::_Select1st<std::pair<doris::DataDir* const, std::vector<long, std::allocator<long> > > >, std::less<doris::DataDir*>, std::allocator<std::pair<doris::DataDir* const, std::vector<long, std::allocator<long> > > > >::_M_begin() /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/stl_tree.h:737
...
```
**To Reproduce**
It's hard to reproduce the bug... but I found a way to stabilize the recurrence problem ....
we could modify be/service/doris_main.cpp:
```
heartbeat_thrift_server = nullptr;
sleep(20); // modify here
doris::ExecEnv::destroy(exec_env);
return 0;
```
1. exec ./bin/start_be.sh
2. kill be
It seems that when StorageEngine is deleted , but the bachground thread is still runting, when background thread try to access StorageEngine ... BE will crash
**Expected behavior**
BE shouldn't exit with segmentfault,
**Desktop (please complete the following information):**
- OS: CentOS 6
** Some Solution **
make StorageEngine extends shared_from_this
or
wait backgroud exit before StorageEngine destroyed
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain commented on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain commented on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-757094232
@acelyc111
if i add some sleep code ,it also core here:
```
#0 0x00007f6e943323f7 in raise () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#1 0x00007f6e943337d8 in abort () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#2 0x0000000001be6256 in google::DumpStackTraceAndExit () at src/utilities.cc:147
#3 0x0000000001bdd55d in google::LogMessage::Fail () at src/logging.cc:1599
#4 0x0000000001bdf3e4 in google::LogMessage::SendToLog (this=0x7f6e213446d0) at src/logging.cc:1553
#5 0x0000000001bdd084 in google::LogMessage::Flush (this=0x7f6e213446d0) at src/logging.cc:1422
#6 0x0000000001bdfe19 in google::LogMessageFatal::~LogMessageFatal (this=<optimized out>, __in_chrg=<optimized out>)
at src/logging.cc:2125
#7 0x00000000011f5d7d in doris::Mutex::lock (this=this@entry=0x6e86d18) at ../src/util/mutex.cpp:107
#8 0x0000000000e7625f in doris::MutexLock::MutexLock (try_lock=false, mutex=0x6e86d18, this=<synthetic pointer>)
at ../src/util/mutex.h:60
#9 doris::CountDownLatch::wait_until (when=..., this=0x6e86d18) at ../src/util/countdown_latch.h:76
#10 doris::CountDownLatch::wait_for (delta=..., this=0x6e86d18) at ../src/util/countdown_latch.h:87
#11 doris::StorageEngine::_compaction_tasks_producer_callback (this=0x6e86c00) at ../src/olap/olap_server.cpp:382
#12 0x00000000011fabe8 in std::function<void ()>::operator()() const (this=0x7b846a8)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/std_function.h:706
#13 doris::Thread::supervise_thread (arg=0x7b84690) at ../src/util/thread.cpp:386
#14 0x00007f6e94dc11c3 in start_thread () from /opt/compiler/gcc-4.8.2/lib64/libpthread.so.0
#15 0x00007f6e943e412d in clone () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] acelyc111 commented on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-756739354
@stdpain what version are you using?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain commented on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain commented on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-756740082
@acelyc111 master commit-id (f6881d2f7b49aca1d0bc90b067eb9e1e73c409d1)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain edited a comment on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain edited a comment on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-757094232
@acelyc111
if i add some sleep code ,it also core here:
```
#0 0x00007f6e943323f7 in raise () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#1 0x00007f6e943337d8 in abort () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#2 0x0000000001be6256 in google::DumpStackTraceAndExit () at src/utilities.cc:147
#3 0x0000000001bdd55d in google::LogMessage::Fail () at src/logging.cc:1599
#4 0x0000000001bdf3e4 in google::LogMessage::SendToLog (this=0x7f6e213446d0) at src/logging.cc:1553
#5 0x0000000001bdd084 in google::LogMessage::Flush (this=0x7f6e213446d0) at src/logging.cc:1422
#6 0x0000000001bdfe19 in google::LogMessageFatal::~LogMessageFatal (this=<optimized out>, __in_chrg=<optimized out>)
at src/logging.cc:2125
#7 0x00000000011f5d7d in doris::Mutex::lock (this=this@entry=0x6e86d18) at ../src/util/mutex.cpp:107
#8 0x0000000000e7625f in doris::MutexLock::MutexLock (try_lock=false, mutex=0x6e86d18, this=<synthetic pointer>)
at ../src/util/mutex.h:60
#9 doris::CountDownLatch::wait_until (when=..., this=0x6e86d18) at ../src/util/countdown_latch.h:76
#10 doris::CountDownLatch::wait_for (delta=..., this=0x6e86d18) at ../src/util/countdown_latch.h:87
#11 doris::StorageEngine::_compaction_tasks_producer_callback (this=0x6e86c00) at ../src/olap/olap_server.cpp:382
#12 0x00000000011fabe8 in std::function<void ()>::operator()() const (this=0x7b846a8)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/std_function.h:706
#13 doris::Thread::supervise_thread (arg=0x7b84690) at ../src/util/thread.cpp:386
#14 0x00007f6e94dc11c3 in start_thread () from /opt/compiler/gcc-4.8.2/lib64/libpthread.so.0
#15 0x00007f6e943e412d in clone () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
```
I use shard_ptr solve it #5214
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain commented on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain commented on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-756740082
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] acelyc111 commented on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-756743172
maybe this patch https://github.com/apache/incubator-doris/pull/5124 can fix it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] acelyc111 commented on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-756739354
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain edited a comment on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain edited a comment on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-757094232
@acelyc111
I apply patch #5124 , and add some sleep code, it also has a dcheck fail
here is a core dump:
```
#0 0x00007f6e943323f7 in raise () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#1 0x00007f6e943337d8 in abort () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#2 0x0000000001be6256 in google::DumpStackTraceAndExit () at src/utilities.cc:147
#3 0x0000000001bdd55d in google::LogMessage::Fail () at src/logging.cc:1599
#4 0x0000000001bdf3e4 in google::LogMessage::SendToLog (this=0x7f6e213446d0) at src/logging.cc:1553
#5 0x0000000001bdd084 in google::LogMessage::Flush (this=0x7f6e213446d0) at src/logging.cc:1422
#6 0x0000000001bdfe19 in google::LogMessageFatal::~LogMessageFatal (this=<optimized out>, __in_chrg=<optimized out>)
at src/logging.cc:2125
#7 0x00000000011f5d7d in doris::Mutex::lock (this=this@entry=0x6e86d18) at ../src/util/mutex.cpp:107
#8 0x0000000000e7625f in doris::MutexLock::MutexLock (try_lock=false, mutex=0x6e86d18, this=<synthetic pointer>)
at ../src/util/mutex.h:60
#9 doris::CountDownLatch::wait_until (when=..., this=0x6e86d18) at ../src/util/countdown_latch.h:76
#10 doris::CountDownLatch::wait_for (delta=..., this=0x6e86d18) at ../src/util/countdown_latch.h:87
#11 doris::StorageEngine::_compaction_tasks_producer_callback (this=0x6e86c00) at ../src/olap/olap_server.cpp:382
#12 0x00000000011fabe8 in std::function<void ()>::operator()() const (this=0x7b846a8)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/std_function.h:706
#13 doris::Thread::supervise_thread (arg=0x7b84690) at ../src/util/thread.cpp:386
#14 0x00007f6e94dc11c3 in start_thread () from /opt/compiler/gcc-4.8.2/lib64/libpthread.so.0
#15 0x00007f6e943e412d in clone () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
```
I use shard_ptr solve it #5214
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain edited a comment on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain edited a comment on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-757094232
@acelyc111
I apply patch #5124 and add some sleep code, it also core with a DCHECK fail
here is a core dump:
```
#0 0x00007f6e943323f7 in raise () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#1 0x00007f6e943337d8 in abort () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#2 0x0000000001be6256 in google::DumpStackTraceAndExit () at src/utilities.cc:147
#3 0x0000000001bdd55d in google::LogMessage::Fail () at src/logging.cc:1599
#4 0x0000000001bdf3e4 in google::LogMessage::SendToLog (this=0x7f6e213446d0) at src/logging.cc:1553
#5 0x0000000001bdd084 in google::LogMessage::Flush (this=0x7f6e213446d0) at src/logging.cc:1422
#6 0x0000000001bdfe19 in google::LogMessageFatal::~LogMessageFatal (this=<optimized out>, __in_chrg=<optimized out>)
at src/logging.cc:2125
#7 0x00000000011f5d7d in doris::Mutex::lock (this=this@entry=0x6e86d18) at ../src/util/mutex.cpp:107
#8 0x0000000000e7625f in doris::MutexLock::MutexLock (try_lock=false, mutex=0x6e86d18, this=<synthetic pointer>)
at ../src/util/mutex.h:60
#9 doris::CountDownLatch::wait_until (when=..., this=0x6e86d18) at ../src/util/countdown_latch.h:76
#10 doris::CountDownLatch::wait_for (delta=..., this=0x6e86d18) at ../src/util/countdown_latch.h:87
#11 doris::StorageEngine::_compaction_tasks_producer_callback (this=0x6e86c00) at ../src/olap/olap_server.cpp:382
#12 0x00000000011fabe8 in std::function<void ()>::operator()() const (this=0x7b846a8)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/std_function.h:706
#13 doris::Thread::supervise_thread (arg=0x7b84690) at ../src/util/thread.cpp:386
#14 0x00007f6e94dc11c3 in start_thread () from /opt/compiler/gcc-4.8.2/lib64/libpthread.so.0
#15 0x00007f6e943e412d in clone () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
```
I have use shard_ptr solve it #5214
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] acelyc111 commented on issue #5213: [BUG] BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
acelyc111 commented on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-757100717
> @acelyc111
> I apply patch #5124 and add some sleep code, it also core with a DCHECK fail
> here is a core dump:
>
> ```
> #0 0x00007f6e943323f7 in raise () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
> #1 0x00007f6e943337d8 in abort () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
> #2 0x0000000001be6256 in google::DumpStackTraceAndExit () at src/utilities.cc:147
> #3 0x0000000001bdd55d in google::LogMessage::Fail () at src/logging.cc:1599
> #4 0x0000000001bdf3e4 in google::LogMessage::SendToLog (this=0x7f6e213446d0) at src/logging.cc:1553
> #5 0x0000000001bdd084 in google::LogMessage::Flush (this=0x7f6e213446d0) at src/logging.cc:1422
> #6 0x0000000001bdfe19 in google::LogMessageFatal::~LogMessageFatal (this=<optimized out>, __in_chrg=<optimized out>)
> at src/logging.cc:2125
> #7 0x00000000011f5d7d in doris::Mutex::lock (this=this@entry=0x6e86d18) at ../src/util/mutex.cpp:107
> #8 0x0000000000e7625f in doris::MutexLock::MutexLock (try_lock=false, mutex=0x6e86d18, this=<synthetic pointer>)
> at ../src/util/mutex.h:60
> #9 doris::CountDownLatch::wait_until (when=..., this=0x6e86d18) at ../src/util/countdown_latch.h:76
> #10 doris::CountDownLatch::wait_for (delta=..., this=0x6e86d18) at ../src/util/countdown_latch.h:87
> #11 doris::StorageEngine::_compaction_tasks_producer_callback (this=0x6e86c00) at ../src/olap/olap_server.cpp:382
> #12 0x00000000011fabe8 in std::function<void ()>::operator()() const (this=0x7b846a8)
> at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/std_function.h:706
> #13 doris::Thread::supervise_thread (arg=0x7b84690) at ../src/util/thread.cpp:386
> #14 0x00007f6e94dc11c3 in start_thread () from /opt/compiler/gcc-4.8.2/lib64/libpthread.so.0
> #15 0x00007f6e943e412d in clone () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
> ```
>
> I have use shard_ptr solve it #5214
@stdpain #5124 is not well fix it, we should join the compaction thread in StorageEngine's stop() function :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain edited a comment on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain edited a comment on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-757094232
@acelyc111
I apply patch #5124 and add some sleep code, it also core with a DCHECK fail
here is a core dump:
```
#0 0x00007f6e943323f7 in raise () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#1 0x00007f6e943337d8 in abort () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
#2 0x0000000001be6256 in google::DumpStackTraceAndExit () at src/utilities.cc:147
#3 0x0000000001bdd55d in google::LogMessage::Fail () at src/logging.cc:1599
#4 0x0000000001bdf3e4 in google::LogMessage::SendToLog (this=0x7f6e213446d0) at src/logging.cc:1553
#5 0x0000000001bdd084 in google::LogMessage::Flush (this=0x7f6e213446d0) at src/logging.cc:1422
#6 0x0000000001bdfe19 in google::LogMessageFatal::~LogMessageFatal (this=<optimized out>, __in_chrg=<optimized out>)
at src/logging.cc:2125
#7 0x00000000011f5d7d in doris::Mutex::lock (this=this@entry=0x6e86d18) at ../src/util/mutex.cpp:107
#8 0x0000000000e7625f in doris::MutexLock::MutexLock (try_lock=false, mutex=0x6e86d18, this=<synthetic pointer>)
at ../src/util/mutex.h:60
#9 doris::CountDownLatch::wait_until (when=..., this=0x6e86d18) at ../src/util/countdown_latch.h:76
#10 doris::CountDownLatch::wait_for (delta=..., this=0x6e86d18) at ../src/util/countdown_latch.h:87
#11 doris::StorageEngine::_compaction_tasks_producer_callback (this=0x6e86c00) at ../src/olap/olap_server.cpp:382
#12 0x00000000011fabe8 in std::function<void ()>::operator()() const (this=0x7b846a8)
at /ssd1/opt/stdpain/workspace/doris/workspace/doris-toolchain/gcc730/include/c++/7.3.0/bits/std_function.h:706
#13 doris::Thread::supervise_thread (arg=0x7b84690) at ../src/util/thread.cpp:386
#14 0x00007f6e94dc11c3 in start_thread () from /opt/compiler/gcc-4.8.2/lib64/libpthread.so.0
#15 0x00007f6e943e412d in clone () from /opt/compiler/gcc-4.8.2/lib64/libc.so.6
```
I use shard_ptr solve it #5214
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain commented on issue #5213: [BUG] BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain commented on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-757126787
@acelyc111
> [@stdpain](https://github.com/stdpain) [#5124](https://github.com/apache/incubator-doris/pull/5124) is not well fix it, we should join the compaction thread in StorageEngine's stop() function :)
pass a shared_ptr to the compaction thread , so we don't have to join compaction thread
```c++
[this]() { this->shared_from_this()->_compaction_tasks_producer_callback(); }
```
'this' pointer won't be freed unless all background thread exited
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain edited a comment on issue #5213: BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain edited a comment on issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213#issuecomment-757094232
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org
[GitHub] [incubator-doris] stdpain closed issue #5213: [BUG] BE may probabilistic trigger segmentfault when BE exit
Posted by GitBox <gi...@apache.org>.
stdpain closed issue #5213:
URL: https://github.com/apache/incubator-doris/issues/5213
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org