You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Hongshun Wang (Jira)" <ji...@apache.org> on 2024/07/17 10:00:00 UTC
[jira] [Commented] (FLINK-35859) [flink-cdc] Fix: The assigner is not ready to offer finished split information, this should not be called
[ https://issues.apache.org/jira/browse/FLINK-35859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17866677#comment-17866677 ]
Hongshun Wang commented on FLINK-35859:
---------------------------------------
I'd like to do it.
> [flink-cdc] Fix: The assigner is not ready to offer finished split information, this should not be called
> ---------------------------------------------------------------------------------------------------------
>
> Key: FLINK-35859
> URL: https://issues.apache.org/jira/browse/FLINK-35859
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.1.1
> Reporter: Hongshun Wang
> Priority: Minor
> Fix For: cdc-3.2.0
>
>
> When use CDC with newly added table, an error occurs:
> {code:java}
> The assigner is not ready to offer finished split information, this should not be called. {code}
> It's because:
> 1. when stop then restart the job , the status is NEWLY_ADDED_ASSIGNING_SNAPSHOT_FINISHED.
>
> 2. Then Enumerator will send each reader with
> BinlogSplitUpdateRequestEvent to update binlog. (see org.apache.flink.cdc.connectors.mysql.source.enumerator.MySqlSourceEnumerator#syncWithReaders).
> 3. The Reader will suspend binlog reader then send BinlogSplitMetaRequestEvent to Enumerator.
> 4. The Enumerator found that some tables are not sent, an error will occur
> {code:java}
> private void sendBinlogMeta(int subTask, BinlogSplitMetaRequestEvent requestEvent) {
> // initialize once
> if (binlogSplitMeta == null) {
> final List<FinishedSnapshotSplitInfo> finishedSnapshotSplitInfos =
> splitAssigner.getFinishedSplitInfos();
> if (finishedSnapshotSplitInfos.isEmpty()) {
> LOG.error(
> "The assigner offers empty finished split information, this should not happen");
> throw new FlinkRuntimeException(
> "The assigner offers empty finished split information, this should not happen");
> }
> binlogSplitMeta =
> Lists.partition(
> finishedSnapshotSplitInfos, sourceConfig.getSplitMetaGroupSize());
> }
> }{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)