[jira] [Updated] (BEAM-12220) ZipFiles.zipDirectory leaks native JVM memory

["Kenneth Knowles (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BEAM-12220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kenneth Knowles updated BEAM-12220:
-----------------------------------
    Status: Open  (was: Triage Needed)

> ZipFiles.zipDirectory leaks native JVM memory
> ---------------------------------------------
>
>                 Key: BEAM-12220
>                 URL: https://issues.apache.org/jira/browse/BEAM-12220
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-core, runner-dataflow, sdk-java-core
>    Affects Versions: 2.22.0
>            Reporter: Brian Hulette
>            Assignee: Brian Hulette
>            Priority: P2
>
> Noticed this in a long-running JVM process starting multiple Dataflow pipelines. Heap usage slowly rose as observed by {{jcmd <pid> VM.native_memory summary.diff}}, but a heap dump did not show a consistent amount of usage.
> Followed [this guide|https://github.com/jeffgriffith/native-jvm-leaks/blob/master/README.md] for identify native memory leaks, which pointed me to java.util.ZipFile.
> We use it in [runners.core-construction.PipelineResources|https://github.com/apache/beam/blob/8e6695643b69da1ab83c9aad16a913cf6e442823/runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/resources/PipelineResources.java#L132] via [sdk.util.ZipFiles|https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ZipFiles.java]
> Closing the ZipOutputStream [here|https://github.com/apache/beam/blob/47cfbcb63f4d0642d26106485bc6fdb894da3086/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ZipFiles.java#L226] seems to alleviate the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)