You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@flink.apache.org by "Wei Zhong (Jira)" <ji...@apache.org> on 2020/11/25 03:54:00 UTC

[jira] [Created] (FLINK-20333) Flink standalone cluster throws metaspace OOM after submitting multiple PyFlink UDF jobs.

Wei Zhong created FLINK-20333:
---------------------------------

             Summary: Flink standalone cluster throws metaspace OOM after submitting multiple PyFlink UDF jobs.
                 Key: FLINK-20333
                 URL: https://issues.apache.org/jira/browse/FLINK-20333
             Project: Flink
          Issue Type: Bug
          Components: API / Python
            Reporter: Wei Zhong


Currently the Flink standalone cluster will throw metaspace OOM after submitting multiple PyFlink UDF jobs. The root cause is that there are many soft references and Finalizers in memory, which prevent the garbage collection of the finished PyFlink job classloader. 

Due to their existence, it needs multiple full gc to reclaim the classloader of the completed job. If only one full gc is performed before the metaspace space is insufficient, then OOM will occur.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)