[jira] [Created] (AVRO-1896) Leaking Deflater Objects

["Chris Bowzer (JIRA)" <ji...@apache.org>]

Chris Bowzer created AVRO-1896:
----------------------------------

             Summary: Leaking Deflater Objects
                 Key: AVRO-1896
                 URL: https://issues.apache.org/jira/browse/AVRO-1896
             Project: Avro
          Issue Type: Improvement
          Components: java
            Reporter: Chris Bowzer
            Priority: Minor


In the java version, with normal creation and use of a DeflateCodec there is a reference to a Deflater object that never has the end method called on it.  While this item still eventually gets garbage collected and release it's c-heap resources, I have run into the issue of a build up of these objects (and the resources they are holding) when using many instances of these objects in parallel.  This has caused a hard to diagnose memory issue where the java heap keeps getting smaller while all resources we have access to are being closed.
While we have solved this issued by recycling DataFileWriters so that there is only ever a limited amount of DeflateCodec instances to be gc'd, this is not an ideal solution. It would be great if there was some way of closing the DeflateCodec instance held inside a DataFileWriter so that deflater.end() can be called explicitly rather than waiting for the finalizer to run.
Thanks!



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)