You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xiqian Yu (Jira)" <ji...@apache.org> on 2026/09/29 06:07:00 UTC
[jira] [Assigned] (FLINK-38061) Caused by: java.util.ConcurrentModificationException When the MySql Chunk Split stage triggers a checkpoint
[ https://issues.apache.org/jira/browse/FLINK-38061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiqian Yu reassigned FLINK-38061:
---------------------------------
Assignee: Jia Fan (was: Yanquan Lv)
> Caused by: java.util.ConcurrentModificationException When the MySql Chunk Split stage triggers a checkpoint
> -----------------------------------------------------------------------------------------------------------
>
> Key: FLINK-38061
> URL: https://issues.apache.org/jira/browse/FLINK-38061
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.4.0
> Reporter: potterhe
> Assignee: Jia Fan
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.7.0
>
>
> {code:java}
> 2025-06-19 15:17:22,914 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - The actual distribution factor for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_127 is 2.3258008172726633E13, the lower bound of evenly distribution factor is 0.05, the upper bound of evenly distribution factor is 1000.0
> 2025-06-19 15:17:22,914 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - Use unevenly-sized chunks for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_127, the chunk size is 50000 from null
> 2025-06-19 15:17:23,002 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering checkpoint 1 (type=CheckpointType{name='Checkpoint', sharingFilesStrategy=FORWARD_BACKWARD}) @ 1750317442997 for job f44cd9bc7a4c8eeec0aa28cb959dcfcf.
> 2025-06-19 15:17:23,034 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - ChunkSplitter has split 0 chunks for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_127
> 2025-06-19 15:17:23,034 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Split table mall_oms_sharding_3.oms_performance_serve_detail_sharding_127 into 1 chunks, time cost: 146ms.
> 2025-06-19 15:17:23,034 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Start splitting table mall_oms_sharding_3.oms_performance_serve_detail_sharding_96 into chunks...
> 2025-06-19 15:17:23,059 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - The distribution factor of table mall_oms_sharding_3.oms_performance_serve_detail_sharding_96 is 2.358148187707668E13 according to the min split key 612073909416749057, max split key 812115620179990530 and approximate row count 8483
> 2025-06-19 15:17:23,059 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - The actual distribution factor for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_96 is 2.358148187707668E13, the lower bound of evenly distribution factor is 0.05, the upper bound of evenly distribution factor is 1000.0
> 2025-06-19 15:17:23,059 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - Use unevenly-sized chunks for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_96, the chunk size is 50000 from null
> 2025-06-19 15:17:23,135 INFO org.apache.flink.cdc.runtime.operators.schema.common.SchemaRegistry [] - Going to start checkpoint No.1
> 2025-06-19 15:17:23,181 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - ChunkSplitter has split 0 chunks for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_96
> 2025-06-19 15:17:23,181 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Split table mall_oms_sharding_3.oms_performance_serve_detail_sharding_96 into 1 chunks, time cost: 147ms.
> 2025-06-19 15:17:23,181 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Start splitting table mall_oms_sharding_3.oms_performance_serve_detail_sharding_97 into chunks...
> 2025-06-19 15:17:23,208 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - The distribution factor of table mall_oms_sharding_3.oms_performance_serve_detail_sharding_97 is 1.9248231912303316E13 according to the min split key 612073859831687169, max split key 812332464647290882 and approximate row count 10404
> 2025-06-19 15:17:23,208 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - The actual distribution factor for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_97 is 1.9248231912303316E13, the lower bound of evenly distribution factor is 0.05, the upper bound of evenly distribution factor is 1000.0
> 2025-06-19 15:17:23,208 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - Use unevenly-sized chunks for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_97, the chunk size is 50000 from null
> 2025-06-19 15:17:23,323 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - ChunkSplitter has split 0 chunks for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_97
> 2025-06-19 15:17:23,323 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Split table mall_oms_sharding_3.oms_performance_serve_detail_sharding_97 into 1 chunks, time cost: 142ms.
> 2025-06-19 15:17:23,323 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Start splitting table mall_oms_sharding_3.oms_performance_serve_detail_sharding_98 into chunks...
> 2025-06-19 15:17:23,351 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - The distribution factor of table mall_oms_sharding_3.oms_performance_serve_detail_sharding_98 is 2.72202276885709E13 according to the min split key 612074076756895745, max split key 812115530040203266 and approximate row count 7349
> 2025-06-19 15:17:23,351 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - The actual distribution factor for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_98 is 2.72202276885709E13, the lower bound of evenly distribution factor is 0.05, the upper bound of evenly distribution factor is 1000.0
> 2025-06-19 15:17:23,351 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - Use unevenly-sized chunks for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_98, the chunk size is 50000 from null
> 2025-06-19 15:17:23,476 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlChunkSplitter [] - ChunkSplitter has split 0 chunks for table mall_oms_sharding_3.oms_performance_serve_detail_sharding_98
> 2025-06-19 15:17:23,477 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Split table mall_oms_sharding_3.oms_performance_serve_detail_sharding_98 into 1 chunks, time cost: 154ms.
> 2025-06-19 15:17:23,477 INFO org.apache.flink.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner [] - Start splitting table mall_oms_sharding_3.oms_performance_serve_detail_sharding_99 into chunks...
> 2025-06-19 15:17:23,479 WARN org.apache.flink.runtime.checkpoint.CheckpointFailureManager [] - Failed to trigger or complete checkpoint 1 for job f44cd9bc7a4c8eeec0aa28cb959dcfcf. (0 consecutive failed attempts so far)
> org.apache.flink.runtime.checkpoint.CheckpointException: Trigger checkpoint failure.
> at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.lambda$getCheckpointException$17(CheckpointCoordinator.java:2365) ~[flink-dist-1.20.1.jar:1.20.1]
> at java.util.Optional.orElseGet(Unknown Source) ~[?:?]
> at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.getCheckpointException(CheckpointCoordinator.java:2364) ~[flink-dist-1.20.1.jar:1.20.1]
> at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.onTriggerFailure(CheckpointCoordinator.java:1056) ~[flink-dist-1.20.1.jar:1.20.1]
> at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.onTriggerFailure(CheckpointCoordinator.java:1034) ~[flink-dist-1.20.1.jar:1.20.1]
> at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.lambda$startTriggeringCheckpoint$7(CheckpointCoordinator.java:760) ~[flink-dist-1.20.1.jar:1.20.1]
> at java.util.concurrent.CompletableFuture.uniHandle(Unknown Source) ~[?:?]
> at java.util.concurrent.CompletableFuture$UniHandle.tryFire(Unknown Source) ~[?:?]
> at java.util.concurrent.CompletableFuture$Completion.run(Unknown Source) ~[?:?]
> at org.apache.flink.util.MdcUtils.lambda$wrapRunnable$1(MdcUtils.java:70) ~[flink-dist-1.20.1.jar:1.20.1]
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) [?:?]
> at java.util.concurrent.FutureTask.run(Unknown Source) [?:?]
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) [?:?]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:?]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:?]
> at java.lang.Thread.run(Unknown Source) [?:?]
> Caused by: java.util.ConcurrentModificationException
> at java.util.HashMap$HashIterator.nextNode(Unknown Source) ~[?:?]
> at java.util.HashMap$EntryIterator.next(Unknown Source) ~[?:?]
> at java.util.HashMap$EntryIterator.next(Unknown Source) ~[?:?]
> at org.apache.flink.cdc.connectors.mysql.source.split.MySqlSplitSerializer.writeTableSchemas(MySqlSplitSerializer.java:194) ~[flink-cdc-pipeline-connector-mysql-3.4.0.jar:3.5-SNAPSHOT]
> at org.apache.flink.cdc.connectors.mysql.source.assigners.state.PendingSplitsStateSerializer.serializeSnapshotPendingSplitsState(PendingSplitsStateSerializer.java:165) ~[flink-cdc-pipeline-connector-mysql-3.4.0.jar:3.5-SNAPSHOT]
> at org.apache.flink.cdc.connectors.mysql.source.assigners.state.PendingSplitsStateSerializer.serializeHybridPendingSplitsState(PendingSplitsStateSerializer.java:182) ~[flink-cdc-pipeline-connector-mysql-3.4.0.jar:3.5-SNAPSHOT]
> at org.apache.flink.cdc.connectors.mysql.source.assigners.state.PendingSplitsStateSerializer.serialize(PendingSplitsStateSerializer.java:85) ~[flink-cdc-pipeline-connector-mysql-3.4.0.jar:3.5-SNAPSHOT]
> at org.apache.flink.cdc.connectors.mysql.source.assigners.state.PendingSplitsStateSerializer.serialize(PendingSplitsStateSerializer.java:45) ~[flink-cdc-pipeline-connector-mysql-3.4.0.jar:3.5-SNAPSHOT]
> at org.apache.flink.runtime.source.coordinator.SourceCoordinator.writeCheckpointBytes(SourceCoordinator.java:594) ~[flink-dist-1.20.1.jar:1.20.1]
> at org.apache.flink.runtime.source.coordinator.SourceCoordinator.toBytes(SourceCoordinator.java:579) ~[flink-dist-1.20.1.jar:1.20.1]
> at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$checkpointCoordinator$7(SourceCoordinator.java:424) ~[flink-dist-1.20.1.jar:1.20.1]
> at org.apache.flink.runtime.source.coordinator.SourceCoordinator.lambda$runInEventLoop$10(SourceCoordinator.java:533) ~[flink-dist-1.20.1.jar:1.20.1]
> at org.apache.flink.util.ThrowableCatchingRunnable.run(ThrowableCatchingRunnable.java:40) ~[flink-dist-1.20.1.jar:1.20.1]
> ... 6 more
> 2025-06-19 15:17:23,484 INFO org.apache.flink.runtime.source.coordinator.SourceCoordinator [] - Marking checkpoint 1 as aborted for source Source: Flink CDC Event Source: mysql. {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)