You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iggy.apache.org by "mattp5657 (via GitHub)" <gi...@apache.org> on 2026/10/26 01:13:47 UTC
[PR] feat(connectors): add MSSQL CDC source connector [iggy]
mattp5657 opened a new pull request, #4648:
URL: https://github.com/apache/iggy/pull/4648
## Which issue does this PR address?
Closes #3573
## Rationale
Iggy has no SQL Server source. This adds a CDC source, starting from the project lsabi shared on the issue and rebuilt on the current SDK so that no change is skipped silently.
## What changed?
Adds `core/connectors/sources/mssql_source/`, which reads SQL Server CDC change tables and emits one JSON event per insert, update or delete:
- Events from all configured tables are merged in commit order. A batch never splits an update's before and after rows.
- The checkpoint is committed only on ACK, and message ids are deterministic, so a re-read carries the same ids.
- The source stops instead of skipping changes on a cleanup gap, a dropped capture instance, CDC being disabled, or a restored database.
- A change that can't be converted holds progress and is logged once a minute.
The plugin is also added to the edge release lists. There are 31 integration tests against SQL Server 2022 in a shared container.
**Out of scope:** no initial snapshot, one database per connector, and capture instance switches only at startup. `sql_variant` and CLR types are rejected. The pre-2022 `commit_time` fallback and availability group failover are untested.
## Local Execution
- Passed: fmt, clippy, 112 unit tests, 31 integration tests
- Pre-commit hooks ran
## AI Usage
1. **Tools:** Claude
2. **Use:** Scaffolding, implementation and iteration on review findings.
3. **Verification:** Local compilation, unit tests and the integration suite against a live SQL Server container.
4. Reviewed and can explain every line if asked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add MSSQL CDC source connector [iggy]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #4648:
URL: https://github.com/apache/iggy/pull/4648#issuecomment-6406038027
<!-- iggy-pr-triage-welcome -->
Thanks for the PR. It is labeled `S-waiting-on-review` and queued for review.
If AI was used to create this PR, you must run the [`team-review` skill](https://github.com/apache/iggy/blob/master/.claude/skills/team-review/SKILL.md) before opening it.
Slash commands (own line, regular comment) move it around the queue:
- `/ready` - back to `S-waiting-on-review` after addressing feedback
- `/author` - flip to `S-waiting-on-author` while you finish changes
- `/request-review @user-or-team` - request a reviewer
- `/pin` - exempt the PR from the stale bot, `/unpin` to undo
See [CONTRIBUTING.md](https://github.com/apache/iggy/blob/master/CONTRIBUTING.md#pr-triage-commands) for details.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(connectors): add MSSQL CDC source connector [iggy]
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #4648:
URL: https://github.com/apache/iggy/pull/4648#issuecomment-6406324235
## [Codecov](https://app.codecov.io/gh/apache/iggy/pull/4648?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `91.77102%` with `184 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 90.77%. Comparing base ([`8b3b54c`](https://app.codecov.io/gh/apache/iggy/commit/8b3b54c554fb4ecbcddff0a7439197c5f9e3b465?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`01e3407`](https://app.codecov.io/gh/apache/iggy/commit/01e340716762328dfd703bfaeee7503c68612880?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 37 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/iggy/pull/4648?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [core/connectors/sources/mssql\_source/src/client.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Fclient.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9jbGllbnQucnM=) | 84.24% | [23 Missing and 43 partials :warning: ](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [core/connectors/sources/mssql\_source/src/lib.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Flib.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9saWIucnM=) | 93.34% | [41 Missing and 23 partials :warning: ](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [core/connectors/sources/mssql\_source/src/error.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Ferror.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9lcnJvci5ycw==) | 80.62% | [27 Missing and 10 partials :warning: ](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [core/connectors/sources/mssql\_source/src/encode.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Fencode.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9lbmNvZGUucnM=) | 97.94% | [0 Missing and 11 partials :warning: ](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [core/connectors/sources/mssql\_source/src/lsn.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Flsn.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9sc24ucnM=) | 92.30% | [3 Missing and 2 partials :warning: ](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [core/connectors/sources/mssql\_source/src/table.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Ftable.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy90YWJsZS5ycw==) | 98.46% | [1 Missing :warning: ](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #4648 +/- ##
============================================
+ Coverage 89.75% 90.77% +1.01%
- Complexity 2145 2989 +844
============================================
Files 1343 1465 +122
Lines 291586 350398 +58812
Branches 247820 294980 +47160
============================================
+ Hits 261727 318075 +56348
- Misses 24131 25582 +1451
- Partials 5728 6741 +1013
```
| [Components](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [Rust Core](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `92.08% <91.77%> (+1.07%)` | :arrow_up: |
| [Java SDK](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `77.78% <ø> (+4.61%)` | :arrow_up: |
| [C# SDK](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.54% <ø> (+2.13%)` | :arrow_up: |
| [Python SDK](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.69% <ø> (-5.03%)` | :arrow_down: |
| [PHP SDK](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `90.67% <ø> (+2.79%)` | :arrow_up: |
| [Node SDK](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `96.45% <ø> (+1.29%)` | :arrow_up: |
| [Go SDK](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.13% <ø> (-1.76%)` | :arrow_down: |
| [C++ SDK](https://app.codecov.io/gh/apache/iggy/pull/4648/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `85.51% <ø> (+0.81%)` | :arrow_up: |
| [Files with missing lines](https://app.codecov.io/gh/apache/iggy/pull/4648?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [core/connectors/sources/mssql\_source/src/table.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Ftable.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy90YWJsZS5ycw==) | `98.46% <98.46%> (ø)` | |
| [core/connectors/sources/mssql\_source/src/lsn.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Flsn.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9sc24ucnM=) | `92.30% <92.30%> (ø)` | |
| [core/connectors/sources/mssql\_source/src/encode.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Fencode.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9lbmNvZGUucnM=) | `97.94% <97.94%> (ø)` | |
| [core/connectors/sources/mssql\_source/src/error.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Ferror.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9lcnJvci5ycw==) | `80.62% <80.62%> (ø)` | |
| [core/connectors/sources/mssql\_source/src/lib.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Flib.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9saWIucnM=) | `93.34% <93.34%> (ø)` | |
| [core/connectors/sources/mssql\_source/src/client.rs](https://app.codecov.io/gh/apache/iggy/pull/4648?src=pr&el=tree&filepath=core%2Fconnectors%2Fsources%2Fmssql_source%2Fsrc%2Fclient.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-Y29yZS9jb25uZWN0b3JzL3NvdXJjZXMvbXNzcWxfc291cmNlL3NyYy9jbGllbnQucnM=) | `84.24% <84.24%> (ø)` | |
... and [573 files with indirect coverage changes](https://app.codecov.io/gh/apache/iggy/pull/4648/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@iggy.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org