You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "david-streamlio (via GitHub)" <gi...@apache.org> on 2026/07/09 20:05:14 UTC
[I] [feat][io] Add Debezium CockroachDB source connector [pulsar-connectors]
david-streamlio opened a new issue, #68:
URL: https://github.com/apache/pulsar-connectors/issues/68
## Motivation
Debezium ships a CockroachDB source connector (`io.debezium:debezium-connector-cockroachdb`, present in the `debezium-bom` at our pinned version), but this repository does not wrap it. We currently wrap 5 of Debezium's 12 source connectors: MySQL, PostgreSQL, MongoDB, Oracle, and SQL Server.
Unlike the other gaps, this connector does **not** use the logical-decoding or binlog machinery — it consumes CockroachDB's native **changefeeds**.
## Proposal
Add a `debezium/cockroachdb` module (`pulsar-io-debezium-cockroachdb`) following the structure of the existing Debezium modules:
- `DebeziumCockroachDbSource` extending `DebeziumSource`
- `include("debezium:pulsar-io-debezium-cockroachdb")` in `settings.gradle.kts`
- `debezium-connector-cockroachdb = { module = "io.debezium:debezium-connector-cockroachdb" }` in the version catalog
- NAR packaging + distribution wiring
- A Testcontainers integration test using the `cockroachdb/cockroach` image (`start-single-node --insecure`), enabling rangefeeds (`SET CLUSTER SETTING kv.rangefeed.enabled = true`) and creating a changefeed in setup
## Notes
- **This is the newest and least battle-tested connector of the set** — it arrived in the 3.4 line. Treat it as lower priority than #MARIADB_ISSUE (MariaDB) and Db2 unless there is concrete user demand, and expect the connector's own behavior to shift across Debezium minors (see the migration tracked in #63).
- Because it uses native changefeeds rather than the standard relational CDC path, verify the emitted record shape and offset semantics match what `DebeziumSource` and `PulsarSchemaHistory` expect — it may not need schema history at all.
- Use the `GenericContainer` pattern (see the Testcontainers version conflict noted in #62).
_Identified by a scan of the Debezium connector catalog against our connector inventory._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org