Re: [I] pipeline migration: parse binlog error when mysql column type is datetime(4) in increment task [shardingsphere]

["sandynz (via GitHub)" <gi...@apache.org>]

sandynz commented on issue #35531:
URL: https://github.com/apache/shardingsphere/issues/35531#issuecomment-4257683959

   
   ## Problem Understanding
   
   - Issue reports a parsing failure during MySQL migration incremental stage when handling zero-value `datetime(4)`.
   - The critical column is `datetime(4)` with default value `0000-00-00 00:00:00.0000`.
   - Based on existing ShardingSphere documentation and existing time-type migration E2E coverage, zero-value time migration is supposed to be supported, so this is not a usage issue.
   
   ## Root Cause
   
   - In MySQL binlog, `DATETIME2` contains a 5-byte datetime body plus extra fractional bytes determined by precision; `datetime(4)` requires 2 extra bytes.
   - `MySQLDatetime2BinlogProtocolValue` returns `"0000-00-00 00:00:00"` early on the zero-value branch and does not consume those 2 fractional bytes.
   - As a result, the row-event payload cursor becomes misaligned and all following columns are decoded from the wrong position.
   
   ## Analysis
   
   - The real problem is not that the zero datetime value itself is converted incorrectly, but that the binlog row event is not fully consumed.
   - Once `DATETIME2` leaves 2 bytes unread for `datetime(4)`, the column boundary of following fields or the next event is broken.
   - This explains runtime errors such as `MySQLBinlogEventHandler protocol resolution error`, and downstream failures where `DATE` is decoded with invalid values like `month=0` or `day=0`.
   - The fact that the previous `MySQLTimeTypesMigrationE2EIT` could pass does not disprove the bug; it simply did not reliably hit the combination of “incremental stage + zero `datetime(4)` + decoding of following columns”.
   
   ## Conclusion
   
   - This is a real bug, not a misuse case.
   - The direct root cause is that `MySQLDatetime2BinlogProtocolValue` does not consume the fractional bytes for zero-value `datetime(4)`, which misaligns incremental binlog decoding.
   - The correct direction is to fix the zero-value byte-consumption logic in `DATETIME2` decoding and add regression coverage that reliably exercises this scenario.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org