[jira] [Created] (FLINK-28248) Metaspace memory is leaking when repeatedly submitting Beam pipelines via the REST API

["Arkadiusz Gasinski (Jira)" <ji...@apache.org>]

Arkadiusz Gasinski created FLINK-28248:
------------------------------------------

             Summary: Metaspace memory is leaking when repeatedly submitting Beam pipelines via the REST API
                 Key: FLINK-28248
                 URL: https://issues.apache.org/jira/browse/FLINK-28248
             Project: Flink
          Issue Type: Bug
          Components: API / Core
    Affects Versions: 1.14.4
            Reporter: Arkadiusz Gasinski
         Attachments: image-2022-06-24-14-45-51-689.png

We have a Flink cluster running on k8s/OpenShift in session mode running our Apache Beam pipelines. Some of these pipelines are streaming pipelines and run continuously; some are batch pipelines submitted periodically whenever there is a load to be processed.

We believe that the batch pipelines cause the issue. We submit 1 to several batch jobs every 5 minutes. For each job, a new instance of the ChildFirstClassLoader is instantiated and it looks like they are not closed properly after the job finishes.

Attached is the screenshot from the Eclipse memory analyzer - from the Leak Suspects report. When the heap dump was captured, there were 2 streaming and several batch jobs running plus over 100 finished batch jobs.

!image-2022-06-24-14-45-51-689.png!



--
This message was sent by Atlassian Jira
(v8.20.7#820007)