[GitHub] [incubator-druid] RestfulBlue opened a new issue #7505: Auto compaction tasks leaks disk space in case of failure

[GitBox <gi...@apache.org>]

RestfulBlue opened a new issue #7505: Auto compaction tasks leaks disk space in case of failure
URL: https://github.com/apache/incubator-druid/issues/7505
 
 
   Hi, recently we was trying to use auto compaction, but found some strange behavior of temporary data location. At first we make a mistake and use java.io.tmpdir in tmpfs. After some time total amount of compaction data become 50gb. 
   
   It looks like at some time there was no enough ram to fit all the things and tasks starts failing. But with that it looks like after task was failed space was not released. Even when no tasks is running, we was seeing about 48gb of data holded in druid jvm tmp dir. 
   
   After that we disable auto compaction and switch druid tmp dir to somethere else. But auto compaction is really important for us. for example we have tables with 3 thousands of segments in tables with 1gb total :(
   
   In other words, it looks like compaction tasks leaks disk space in case of failure. Is there a location, where we can point java.io.tmpdir and be sure space will not be leaked? 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org