You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Gengliang Wang (Jira)" <ji...@apache.org> on 2026/05/03 22:43:00 UTC
[jira] [Created] (SPARK-56711) CDC: Restricting data type of _commit_version to Long / String
Gengliang Wang created SPARK-56711:
--------------------------------------
Summary: CDC: Restricting data type of _commit_version to Long / String
Key: SPARK-56711
URL: https://issues.apache.org/jira/browse/SPARK-56711
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.2.0
Reporter: Gengliang Wang
Assignee: Gengliang Wang
1. Every realistic CDC source falls into one of two camps — a monotonic numeric version (Delta, Iceberg, LSN, Kafka offset → Long) or an opaque identifier (commit hashes,composite IDs → String). The other allowed types are either strict subsets (Integer⊂ Long) or duplicate the role of _commit_timestamp (Timestamp), and Float/Double/Decimal/Boolean/Binary just add NaN/precision/ordering foot-guns with no expressive power gained.
2. A narrower contract is a one-way door we can always relax later. Going from "Long+String" to "any AtomicType" is non-breaking; the reverse breaks connectors. Locking down now while there are zero external connectors costs nothing.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org