[GitHub] [incubator-doris] gaodayue opened a new pull request #2414: fix several races in stream load that could cause BE crash

[GitBox <gi...@apache.org>]

gaodayue opened a new pull request #2414: fix several races in stream load that could cause BE crash
URL: https://github.com/apache/incubator-doris/pull/2414
 
 
   Fixes #2413 and #1984 
   
   This CL fixes the following problems
   1. check whether TabletsChannel has been closed/cancelled in `reduce_mem_usage` to avoid using a closed DeltaWriter
   2. make `FlushHandle.wait` wait for all submitted tasks to finish so that memtable is deallocated before its delta writer
   3. make `~MemTracker()` release its consumption bytes to accommodate situations in aggregate_func.h that bitmap and hll call `MemTracker::consume` without corresponding `MemTracker::release`, which cause the consumption of root tracker never drops to zero
   
   Besides I also add various verbose logging that helps me locate these concurrency bugs and hopefully would be helpful for others.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org