[jira] [Resolved] (AVRO-3802) Memory leak on decompressing deflate codec avro files

["Ismaël Mejía (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/AVRO-3802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ismaël Mejía resolved AVRO-3802.
--------------------------------
    Resolution: Fixed

> Memory leak on decompressing deflate codec avro files
> -----------------------------------------------------
>
>                 Key: AVRO-3802
>                 URL: https://issues.apache.org/jira/browse/AVRO-3802
>             Project: Apache Avro
>          Issue Type: Bug
>          Components: csharp
>    Affects Versions: 1.11.1, 1.12.0, 1.11.2, 1.11.3
>            Reporter: amy myo
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.12.0
>
>         Attachments: call tree.PNG, deflate decompression memory profiler.png
>
>          Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> Hi,
> We are using apache.avro C# library for compression and decompression of large avro files. Each file is 2-3MB and decompression of 20 files is using 2.1GB memory and memory keep increasing around 200-300MB for follow up calls. 
> Attached is the memory profiler of three calls. Each call decompress the same 20 files. First call uses 2.1GB , increase to 2.37GB and then 2.68GB. 
> Call tree is showing most of the memory are used by `DeflatCodec.CopyTo` method. 
>  # `CopyTo` method is not using Array pool. Default Microsoft's stream `copyTo` method is much more efficient. 
>  # `inStream` and `outStream` memory streams are not in using statement.
>  # On `outstream` object creation, object can be initialized with defined length/stream capacity. 
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)