[jira] [Comment Edited] (FLINK-31914) Failing to close FlinkKafkaInternalProducer created in KafkaWriter with exactly-once semantic results in memory leak

["datariver (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-31914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17716095#comment-17716095 ] 

datariver edited comment on FLINK-31914 at 4/25/23 6:18 AM:
------------------------------------------------------------

[~chouc] Thanks for the reply, I found that the 1.15.2 version has been fixed.


was (Author: JIRAUSER292566):
Thanks for the reply, I found that the 1.15.2 version has been fixed.

> Failing to close FlinkKafkaInternalProducer created in KafkaWriter with exactly-once semantic results in memory leak
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-31914
>                 URL: https://issues.apache.org/jira/browse/FLINK-31914
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Kafka
>    Affects Versions: 1.15.0
>            Reporter: datariver
>            Priority: Major
>         Attachments: image-2023-04-25-13-47-25-703.png
>
>
> Hi [~arvid] , If Exactly-Once writing is enabled, Kafka's transactional writing will be used. KafkaWriter will create FlinkKafkaInternalProducer in the initialization and snapshotState methods, but there is no place to close it. As Checkpoints increase, Producers will continue to accumulate. Each Producer maintains a Buffer, which will cause memory leaks and Job OOM.
> By dumping an in-memory instance of Task Manager, you can see that there are a lot of Producers:
> !image-2023-04-25-13-47-25-703.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)