[jira] [Created] (PARQUET-353) Compressors not getting recycled while writing parquet files causing memory leak

["Nitin Goyal (JIRA)" <ji...@apache.org>]

Nitin Goyal created PARQUET-353:
-----------------------------------

             Summary: Compressors not getting recycled while writing parquet files causing memory leak
                 Key: PARQUET-353
                 URL: https://issues.apache.org/jira/browse/PARQUET-353
             Project: Parquet
          Issue Type: Bug
          Components: parquet-mr
    Affects Versions: 1.8.0, 1.7.0, 1.6.0
            Reporter: Nitin Goyal


Compressors are not getting recycled while writing parquet files. This is causing native/physical memory leak in my spark app which is parquet write intensive since its creating new compressors everytime i write parquet files.

The actual code issue is that we are creating 'codecFactory' in 'getRecordWriter' method of ParquetOutputFormat.java but not calling codecFactory.release() which is responsible for recycling compressors.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)