You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Yingjie Cao (Jira)" <ji...@apache.org> on 2019/09/20 07:07:00 UTC
[jira] [Commented] (FLINK-14139) Fix potential memory leak of rest
server when using session/standalone cluster
[ https://issues.apache.org/jira/browse/FLINK-14139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16934144#comment-16934144 ]
Yingjie Cao commented on FLINK-14139:
-------------------------------------
Some other user of netty also encountered this problem: [https://github.com/netty/netty/issues/8310].
> Fix potential memory leak of rest server when using session/standalone cluster
> ------------------------------------------------------------------------------
>
> Key: FLINK-14139
> URL: https://issues.apache.org/jira/browse/FLINK-14139
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / REST
> Reporter: Yingjie Cao
> Priority: Major
> Fix For: 1.10.0
>
>
> Flink's rest server uses netty decoder for http request processing and file uploading. However io.netty.handler.codec.http.multipart.DiskAttribute and io.netty.handler.codec.http.multipart.DiskFileUpload class of netty would register some temp files, including post chunks and upload file chunks, to java.io.DeleteOnExitHook which has a potential of memory leak, because the registered file names will never be deleted before the cluster stops.
> Most of the time, this is not a big problem, however we use Flink session cluster a long running service for ad-hoc SQL query, this problem gets worse.
> In fact, Flink handles the clean up of temp files through org.apache.flink.util.ShutdownHookUtil (though not including the post chunks), so it is no need to register these files to java.io.DeleteOnExitHook.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)