[jira] [Created] (DRILL-6482) Fast clean unused direct memory when decompress

["zhoukang (JIRA)" <ji...@apache.org>]

zhoukang created DRILL-6482:
-------------------------------

             Summary: Fast clean unused direct memory when decompress
                 Key: DRILL-6482
                 URL: https://issues.apache.org/jira/browse/DRILL-6482
             Project: Apache Drill
          Issue Type: Improvement
            Reporter: zhoukang


When use *NonBlockedDecompressorStream* which call:

*SnappyDecompressor.setInput*

{code}

public synchronized void setInput(byte[] buffer, int off, int len) {
 SnappyUtil

public synchronized void setInput(byte[] buffer, int off, int len) {
 SnappyUtil.validateBuffer(buffer, off, len);

 if (inputBuffer.capacity() - inputBuffer.position() < len) {
 ByteBuffer newBuffer = ByteBuffer.allocateDirect(inputBuffer.position() + len);
 inputBuffer.rewind();
 newBuffer.put(inputBuffer);
 inputBuffer = newBuffer; 
 } else {
 inputBuffer.limit(inputBuffer.position() + len);
 }
 inputBuffer.put(buffer, off, len);
}

.validateBuffer(buffer, off, len);

 if (inputBuffer.capacity() - inputBuffer.position() < len) {
 ByteBuffer newBuffer = ByteBuffer.allocateDirect(inputBuffer.position() + len);
 inputBuffer.rewind();
 newBuffer.put(inputBuffer);
 inputBuffer = newBuffer; 
 } else {
 inputBuffer.limit(inputBuffer.position() + len);
 }
 inputBuffer.put(buffer, off, len);
}

{code}

If we do not get any full gc for old gen.we may failed by off-heap memory leak



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)