You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xin Gong (Jira)" <ji...@apache.org> on 2024/04/18 04:44:00 UTC
[jira] [Comment Edited] (FLINK-35151) Flink mysql cdc will stuck when suspend binlog split and ChangeEventQueue is full
[ https://issues.apache.org/jira/browse/FLINK-35151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17838452#comment-17838452 ]
Xin Gong edited comment on FLINK-35151 at 4/18/24 4:43 AM:
-----------------------------------------------------------
I get an idea to address this issue by set
currentTaskRunning || queue.remainingCapacity() == 0 for BinlogSplitReader#pollSplitRecords. [~Leonard] PTAL
was (Author: JIRAUSER292212):
I get an idea to address this issue by set
currentTaskRunning || queue.remainingCapacity() == 0 for BinlogSplitReader#pollSplitRecords.
> Flink mysql cdc will stuck when suspend binlog split and ChangeEventQueue is full
> ----------------------------------------------------------------------------------
>
> Key: FLINK-35151
> URL: https://issues.apache.org/jira/browse/FLINK-35151
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Environment: I use master branch reproduce it.
> Reporter: Xin Gong
> Priority: Major
> Attachments: dumpstack.txt
>
>
> Flink mysql cdc will stuck when suspend binlog split and ChangeEventQueue is full.
> Reason is that producing binlog is too fast. MySqlSplitReader#suspendBinlogReaderIfNeed will execute BinlogSplitReader#stopBinlogReadTask to set
> currentTaskRunning to be false after MysqSourceReader receives binlog split update event.
> MySqlSplitReader#pollSplitRecords is executed and
> dataIt is null to execute closeBinlogReader when currentReader is BinlogSplitReader. closeBinlogReader will execute statefulTaskContext.getBinaryLogClient().disconnect(), it could dead lock. Because BinaryLogClient#connectLock is not release when MySqlStreamingChangeEventSource add element to full queue.
>
> You can set StatefulTaskContext#queue to be 1 and run UT NewlyAddedTableITCase#testRemoveAndAddNewTable.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)