You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "Xin Gong (Jira)" <ji...@apache.org> on 2024/09/10 10:13:00 UTC
[jira] [Updated] (FLINK-36247) Potential transaction leak during MySQL snapshot phase
[ https://issues.apache.org/jira/browse/FLINK-36247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xin Gong updated FLINK-36247:
-----------------------------
Description:
copy from
[https://github.com/apache/flink-cdc/issues/1552]
We have found that during the MySQL snapshot phase, a transaction is initiated but never committed, resulting in the transaction being left there for a long time.
For instance, the binlog is like
{code:java}
SET autocommit=0 // a transaction is started by com.ververica.cdc.connectors.mysql.source.utils.StatementUtils#initStatement
SELECT * FROM `database`.`table` WHERE id >= 1111 AND NOT (id = 2222) AND id <= 2222
SHOW MASTER STATUS
SHOW GLOBAL VARIABLES LIKE 'GTID_MODE'
SHOW MASTER STATUS
SELECT @@global.gtid_purgedrollback // the transaction is aborted without a
commit after a long wait {code}
Steps to reproduce the behavior:
1、The test data : Any MySQL database with existing data in the tableThe
1、test code : INSERT INTO any_sink SELECT * FROM mysql_cdc_source_table;
3、The error : A transaction is left without being committed.
From the code, maybe it is the connection.setAutoCommit(false) that causes this issue, but we are still not sure why it is there in the first place.
!https://user-images.githubusercontent.com/5620838/189831936-522c8ae7-a116-4948-8b5d-41b4d0eaab72.png!
was:
copy from
[https://github.com/apache/flink-cdc/issues/1552]
We have found that during the MySQL snapshot phase, a transaction is initiated but never committed, resulting in the transaction being left there for a long time.
For instance, the binlog is like
{code:java}
SET autocommit=0 // a transaction is started by com.ververica.cdc.connectors.mysql.source.utils.StatementUtils#initStatement
SELECT * FROM `database`.`table` WHERE id >= 1111 AND NOT (id = 2222) AND id <= 2222
SHOW MASTER STATUSSHOW GLOBAL VARIABLES LIKE 'GTID_MODE'SHOW MASTER STATUS
SELECT @@global.gtid_purgedrollback // the transaction is aborted without a
commit after a long wait {code}
Steps to reproduce the behavior:
1、The test data : Any MySQL database with existing data in the tableThe
1、test code : INSERT INTO any_sink SELECT * FROM mysql_cdc_source_table;
3、The error : A transaction is left without being committed.
From the code, maybe it is the connection.setAutoCommit(false) that causes this issue, but we are still not sure why it is there in the first place.
!https://user-images.githubusercontent.com/5620838/189831936-522c8ae7-a116-4948-8b5d-41b4d0eaab72.png!
> Potential transaction leak during MySQL snapshot phase
> ------------------------------------------------------
>
> Key: FLINK-36247
> URL: https://issues.apache.org/jira/browse/FLINK-36247
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Reporter: Xin Gong
> Priority: Major
>
> copy from
> [https://github.com/apache/flink-cdc/issues/1552]
>
> We have found that during the MySQL snapshot phase, a transaction is initiated but never committed, resulting in the transaction being left there for a long time.
> For instance, the binlog is like
> {code:java}
> SET autocommit=0 // a transaction is started by com.ververica.cdc.connectors.mysql.source.utils.StatementUtils#initStatement
> SELECT * FROM `database`.`table` WHERE id >= 1111 AND NOT (id = 2222) AND id <= 2222
> SHOW MASTER STATUS
> SHOW GLOBAL VARIABLES LIKE 'GTID_MODE'
> SHOW MASTER STATUS
> SELECT @@global.gtid_purgedrollback // the transaction is aborted without a
> commit after a long wait {code}
> Steps to reproduce the behavior:
> 1、The test data : Any MySQL database with existing data in the tableThe
> 1、test code : INSERT INTO any_sink SELECT * FROM mysql_cdc_source_table;
> 3、The error : A transaction is left without being committed.
> From the code, maybe it is the connection.setAutoCommit(false) that causes this issue, but we are still not sure why it is there in the first place.
>
> !https://user-images.githubusercontent.com/5620838/189831936-522c8ae7-a116-4948-8b5d-41b4d0eaab72.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)