[GitHub] [accumulo] andrewglowacki commented on issue #1011: Slow memory leak

[GitBox <gi...@apache.org>]

andrewglowacki commented on issue #1011: Slow memory leak
URL: https://github.com/apache/accumulo/issues/1011#issuecomment-486768420
 
 
   The issue was DirectByteBuffers (DBB) created by HDFS DFSInputStreams (for RFile reads) not getting cleaned up fast enough (or at all). Setting -XX:MaxDirectMemorySize=2g fixed it.
   
   After quite a bit of looking (multiple weeks), I correlated some multi-hour old/unfreed native memory segments with DBBs in the java heap - sure enough, direct memory usage was very high (multiple GB). In my java heap dump, I found 36k live DBBs with 46 referenced by live DFSInputStreams in the FileManager's openFiles list, 20k cached in the HDFS PacketReceiver 'bufferPool' and the rest awaiting finialization of their parent PacketReceiver.
   
   The sizes were not large, at most 1 MB each, but the number was the issue.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services