You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "wubx (via GitHub)" <gi...@apache.org> on 2025/08/07 02:43:54 UTC

Re: [PR] [Feature][Sink] File support new format: maxwell_json,canal_json,debezium_json (#9278) [seatunnel]

wubx commented on PR #9336:
URL: https://github.com/apache/seatunnel/pull/9336#issuecomment-3162225188

   Thank you very much for your contribution.
   I have test this pr:
   ```
   update t01 set c1='debezium_json' where id=7;
   ```
   debezium format
   ```
   {"before":{"id":7,"c1":"test"},"after":null,"op":"d"}
   {"before":null,"after":{"id":7,"c1":"debezium_json"},"op":"c"}
   ```
   
   maxwell format
   update t01 set c1='debezium_json10' where id=7;
   ```
   {"data":{"id":7,"c1":"debezium_json"},"type":"DELETE"}
   {"data":{"id":7,"c1":"debezium_json10"},"type":"INSERT"}
   ```
   
   
   From the usage, a time field(ts_ms) is required in this format to indicate the generation time of the record. This makes it convenient to obtain the status of the last record.
   
   Full debezium json , source section is also very useful. If the file contains sources that are useful for batch Change Data Capture (CDC) for multiple tables.
   
   ```
   {
       "before": {
           "id": 111,
           "name": "scooter",
           "description": "Big 2-wheel scooter ",
           "weight": 5.18
       },
       "after": {
           "id": 111,
           "name": "scooter",
           "description": "Big 2-wheel scooter ",
           "weight": 5.17
       },
       "source": {
           "version": "1.1.1.Final",
           "connector": "mysql",
           "name": "dbserver1",
           "ts_ms": 1589362330000,
           "snapshot": "false",
           "db": "inventory",
           "table": "products",
           "server_id": 223344,
           "gtid": null,
           "file": "mysql-bin.000003",
           "pos": 2090,
           "row": 0,
           "thread": 2,
           "query": null
       },
       "op": "u",
       "ts_ms": 1589362330904,
       "transaction": null
   }
   ```
   
   Previously, there was no need for a "ts_ms" field at all. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org