You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by "Jake.zhang" <ft...@qq.com> on 2024/10/17 12:17:33 UTC

kafka sink can't recycle FlinkKafkaInternalProducer

Hi all:


flink kafka connector do't close&nbsp;FlinkKafkaInternalProducer when flink checkpoint success in flink 1.20/1.19 .&nbsp; it will create one&nbsp;&nbsp;FlinkKafkaInternalProducer per checkpoint.&nbsp;


what config can fix this ?


3 partition&nbsp; create one producer one checkpoint

回复: kafka sink can't recycle FlinkKafkaInternalProducer

Posted by "Jake.zhang" <ft...@qq.com>.
&nbsp;kafka committer can't recycle kafka writer producer. kafka&nbsp; producer need call `close()`


https://issues.apache.org/jira/browse/FLINK-36569




------------------&nbsp;原始邮件&nbsp;------------------
发件人:                                                                                                                        "Feng Jin"                                                                                    <jinfeng1995@gmail.com&gt;;
发送时间:&nbsp;2024年10月21日(星期一) 晚上10:19
收件人:&nbsp;"Jake.zhang"<ft20082@qq.com&gt;;
抄送:&nbsp;"user"<user@flink.apache.org&gt;;
主题:&nbsp;Re: kafka sink can't recycle FlinkKafkaInternalProducer



FlinkKafkaInternalProducer will close after commit, please confirm if your checkpoint is normally triggered and finished?




Best,
Feng




On Thu, Oct 17, 2024 at 8:20 PM Jake.zhang <ft20082@qq.com&gt; wrote:

Hi all:


flink kafka connector do't close&nbsp;FlinkKafkaInternalProducer when flink checkpoint success in flink 1.20/1.19 .&nbsp; it will create one&nbsp;&nbsp;FlinkKafkaInternalProducer per checkpoint.&nbsp;


what config can fix this ?


3 partition&nbsp; create one producer one checkpoint

Re: kafka sink can't recycle FlinkKafkaInternalProducer

Posted by Feng Jin <ji...@gmail.com>.
FlinkKafkaInternalProducer will close after commit, please confirm if your
checkpoint is normally triggered and finished?


Best,
Feng


On Thu, Oct 17, 2024 at 8:20 PM Jake.zhang <ft...@qq.com> wrote:

> Hi all:
>
> flink kafka connector do't close FlinkKafkaInternalProducer when flink
> checkpoint success in flink 1.20/1.19 .  it will create one  FlinkKafkaInternalProducer
> per checkpoint.
>
> what config can fix this ?
>
> 3 partition  create one producer one checkpoint
>
>
>
>