[jira] [Comment Edited] (FLINK-20885) Exception when use 'canal-json.table.include' to filter Canal binlog but table contains 'source' column

["wangfei (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-20885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17263858#comment-17263858 ] 

wangfei edited comment on FLINK-20885 at 1/13/21, 3:14 AM:
-----------------------------------------------------------

table test_2.userName (int); table test_1.userName (varchar);
 You create a ddl with canal-json.table.include=test_2 .

Then add the binlog of test_1, as in the binlog format above

{"data":[\{"id":"0","username":"zhangsna","password":"1111111","AddTime":"0000-00-00 00:00:00"}],"database":"plink","es":1610507126000,"id":90469,"isDdl":false,"mysqlType":\{"id":"int(11)","username":"varchar(255)","password":"varchar(255)","AddTime":"datetime"},"old":[\{"username":null,"password":null}],"pkNames":["id"],"sql":"","sqlType":\{"id":4,"username":12,"password":12,"AddTime":93},"table":"test_1","ts":1610507127009,"type":"UPDATE"}


 Then you get an error

 


was (Author: wangfeiair2324):
table test_2.userName (int); table test_1.userName (varchar);
You create a canal-json.table.include=test_2
Then you get an error

 

> Exception when use 'canal-json.table.include' to filter Canal binlog but table contains 'source' column
> -------------------------------------------------------------------------------------------------------
>
>                 Key: FLINK-20885
>                 URL: https://issues.apache.org/jira/browse/FLINK-20885
>             Project: Flink
>          Issue Type: Bug
>          Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table SQL / Ecosystem
>            Reporter: Jark Wu
>            Assignee: Nicholas Jiang
>            Priority: Major
>         Attachments: image-2021-01-08-14-52-53-443.png, image-2021-01-08-14-53-40-476.png, screenshot-1.png
>
>
> I found a bug in the canal code. 'canal-json.table.include' does not filter out the binlog of the specified table correctly, which will cause an error in the parsing section. For example, if I want to read the binlog of canal-json.table.include = 'a' table, there is a source field of int in table a, but at this time if table b also has a source field of string, An error will be reported.
>  !screenshot-1.png! 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)