You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by 史 正超 <sh...@outlook.com> on 2020/09/30 02:51:05 UTC
回复: 回复：关于flink sql cdc
HI, Kyle Zhang, 我刚才重现了你的问题，虽然你的mysql binlog设置是ROW格式，但是不排除其它session更改了binlog_format格式。重现步骤：
1. 登录mysql客户端(注意用cmd登录) 执行语句， SET SESSION binlog_format='MIXED'; SET SESSION tx_isolation='REPEATABLE-READ'; COMMIT;
2. 随便update或者insert一条语句。
然后就得到了和你一样的错误：
2020-09-30 10:46:37.607 [debezium-engine] ERROR com.alibaba.ververica.cdc.debezium.DebeziumSourceFunction - Reporting error:
org.apache.kafka.connect.errors.ConnectException: Received DML 'update orders set product_id = 1122 where order_number = 10001' for processing, binlog probably contains events generated with statement or mixed based replication format
at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:207)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:600)
at com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1130)
at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:978)
at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:581)
at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:860)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.connect.errors.ConnectException: Received DML 'update orders set product_id = 1122 where order_number = 10001' for processing, binlog probably contains events generated with statement or mixed based replication format
at io.debezium.connector.mysql.BinlogReader.handleQueryEvent(BinlogReader.java:785)
at io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:583)
... 5 common frames omitted
所以应该是其它session更忙了binlog_format格式，并且事务隔离级别为 REPEATABLE-READ
希望对你有帮助，
best,
shizhengchao
________________________________
发件人: 谢治平 <xi...@163.com>
发送时间: 2020年9月30日 1:25
收件人: user-zh <us...@flink.apache.org>
抄送: user-zh <us...@flink.apache.org>
主题: 回复：关于flink sql cdc
能不能退掉邮箱信息，退出
| |
谢治平
|
|
邮箱：xiezhiping919@163.com
|
签名由 网易邮箱大师 定制
在2020年09月30日 09:24，Kyle Zhang 写道：
show variables like '%binlog_format%'确实是ROW
On Tue, Sep 29, 2020 at 7:39 PM Kyle Zhang <sh...@gmail.com> wrote:
> Hi,all
> 今天在使用sql cdc中遇到以一个问题 ，版本1.11.2，idea中运行，我的ddl是
> CREATE TABLE mysql_binlog (
> id INT NOT NULL,
> emp_name STRING,
> age INT
> ) WITH (
> 'connector' = 'mysql-cdc',
> 'hostname' = 'xxx',
> 'port' = '3306',
> 'username' = 'root',
> 'password' = 'root',
> 'database-name' = 'test',
> 'table-name' = 'empoylee1'
> );
> 结果直接用print table
> 运行一段时间后报错
> 19:22:52,430 ERROR io.debezium.connector.mysql.BinlogReader
> [] - Error during binlog processing. Last offset stored = null,
> binlog reader near position = binlog.001254/132686776
> 19:22:52,430 ERROR io.debezium.connector.mysql.BinlogReader
> [] - Failed due to error: Error processing binlog event
> org.apache.kafka.connect.errors.ConnectException: Received DML 'INSERT
> INTO execution_flows (project_id, flow_id, version, status, submit_time,
> submit_user, update_time) values
> (47,'OfflineComputeTask',1,20,1601378572442,'azkaban',1601378572442)' for
> processing, binlog probably contains events generated with statement or
> mixed based replication format
> at
> io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230)
> ~[flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> at
> io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:207)
> ~[flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> at
> io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:600)
> ~[flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient.notifyEventListeners(BinaryLogClient.java:1130)
> [flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:978)
> [flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:581)
> [flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> at
> com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:860)
> [flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_265]
> Caused by: org.apache.kafka.connect.errors.ConnectException: Received DML
> 'INSERT INTO execution_flows (project_id, flow_id, version, status,
> submit_time, submit_user, update_time) values
> (47,'OfflineComputeTask',1,20,1601378572442,'azkaban',1601378572442)' for
> processing, binlog probably contains events generated with statement or
> mixed based replication format
> at
> io.debezium.connector.mysql.BinlogReader.handleQueryEvent(BinlogReader.java:785)
> ~[flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> at
> io.debezium.connector.mysql.BinlogReader.handleEvent(BinlogReader.java:583)
> ~[flink-connector-mysql-cdc-1.1.0.jar:1.1.0]
> ... 5 more
>
> sql cdc还会解析我其他的表然后报错么？，有没有人遇到类似的问题
>
> Best,
> Kyle Zhang
>