[jira] [Created] (BEAM-12220) ZipFiles.zipDirectory leaks native JVM memory

["Brian Hulette (Jira)" <ji...@apache.org>]

Brian Hulette created BEAM-12220:
------------------------------------

             Summary: ZipFiles.zipDirectory leaks native JVM memory
                 Key: BEAM-12220
                 URL: https://issues.apache.org/jira/browse/BEAM-12220
             Project: Beam
          Issue Type: Bug
          Components: runner-core, runner-dataflow, sdk-java-core
    Affects Versions: 2.22.0
            Reporter: Brian Hulette
            Assignee: Brian Hulette


Noticed this in a long-running JVM process starting multiple Dataflow pipelines. Heap usage slowly rose as observed by {{jcmd <pid> VM.native_memory summary.diff}}, but a heap dump did not show a consistent amount of usage.

Followed [this guide|https://github.com/jeffgriffith/native-jvm-leaks/blob/master/README.md] for identify native memory leaks, which pointed me to java.util.ZipFile.

We use it in [runners.core-construction.PipelineResources|https://github.com/apache/beam/blob/8e6695643b69da1ab83c9aad16a913cf6e442823/runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/resources/PipelineResources.java#L132] via [sdk.util.ZipFiles|https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ZipFiles.java]

Closing the ZipOutputStream [here|https://github.com/apache/beam/blob/47cfbcb63f4d0642d26106485bc6fdb894da3086/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ZipFiles.java#L226] seems to alleviate the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)