[GitHub] [druid] rohangarg commented on pull request #13475: Remove stray reference to fix OOM while merging sketches

[GitBox <gi...@apache.org>]

rohangarg commented on PR #13475:
URL: https://github.com/apache/druid/pull/13475#issuecomment-1337097474

   > There are unit tests in WorkerSketchFetcherTest which test if tasks would be cancelled, but there aren't any tests to see if any references to large objects like key stats are leaked. I suppose we could test something like that with WeakReference?
   I have reproduced the bug and functionally tested this out locally, and these changes seems to fix the bug.
   
   I think doing the functional test is ok for now, maybe in future one way to test it could be to generate multiple small sketches in a test and write them to file. Then we can setup the fetcher for those sketches and check if the fetcher runs stably - it will automatically OOM incase it ends up taking a lot of memory.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org