[jira] [Updated] (HUDI-3774) Fix memory leaks through shutdown hooks

["Alexey Kudinkin (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-3774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Alexey Kudinkin updated HUDI-3774:
----------------------------------
    Priority: Blocker  (was: Major)

> Fix memory leaks through shutdown hooks
> ---------------------------------------
>
>                 Key: HUDI-3774
>                 URL: https://issues.apache.org/jira/browse/HUDI-3774
>             Project: Apache Hudi
>          Issue Type: Bug
>            Reporter: Alexey Kudinkin
>            Priority: Blocker
>         Attachments: Screen Shot 2022-04-01 at 11.58.14 AM.png
>
>
> Our tests occasionally crash w/ OOM in Azure. 
> I re-run the tests locally under the same conditions trying to reproduce it and took a few heap dumps.
> I see that out of about 1.1Gb about 272Mb are the HoodieLogFileReaders retained t/h shutdown hooks. This means that those HoodieLogFileReader has not been properly closed (potentially under conditions that there were some exceptions thrown)
>  
> To make sure that all readers get closed upon reaching their end of life, we'd use try-resource-statement wherever possible, which will guarantee proper closing for the reader under any possible control-flow
> {code:java}
> try (Reader r = new Reader) {
>   // ...
> }
> {code}
>  
> !Screen Shot 2022-04-01 at 11.58.14 AM.png!



--
This message was sent by Atlassian Jira
(v8.20.1#820001)