You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "DanielLeens (via GitHub)" <gi...@apache.org> on 2026/08/23 11:22:20 UTC
[PR] [Test][E2E] Add Zeta CDC cluster failover coverage [seatunnel]
DanielLeens opened a new pull request, #11947:
URL: https://github.com/apache/seatunnel/pull/11947
## Summary
- add an independent engine-side E2E suite for MySQL CDC failover stability in Zeta
- cover regular-cluster, split-cluster, and all-node-down recovery paths with deterministic source/sink reconciliation
- keep the dependency change limited to test-scope MySQL CDC artifacts required by the new engine E2E harness
## Coverage
- cover worker and master failover for a single CDC pipeline in the default two-node cluster
- cover worker and master failover for a single CDC pipeline in a split master/worker cluster
- cover all-node-down restart recovery for both the default cluster and the split cluster
- cover worker and master failover for a two-pipeline CDC job
- verify recovery through job status polling, checkpoint counter growth, and source/sink reconciliation instead of fixed assertion sleeps
## Validation
- ./mvnw -pl seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base spotless:apply
- local compile, unit tests, and E2E runs were not executed because this repository currently relies on GitHub CI for functional validation
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Test][E2E] Add Zeta CDC cluster failover coverage [seatunnel]
Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11947:
URL: https://github.com/apache/seatunnel/pull/11947#issuecomment-5391515229
Follow-up on my previous review: I wrote it about three minutes into this head's (`d6e24425b582`) CI run, while `Build` was still in progress. That run has since finished, and it is now `FAILURE` (fork run `DanielLeens/seatunnel` run `32638154468`), so I want to correct the record rather than let "Ready to merge" stand unqualified.
I pulled the failed job logs (`engine-v2-it` shards 8 and 11) rather than just reading the check name. Both shards fail the same way, and it is not a flake:
```
[ERROR] SeaTunnel : E2E : Engine : Base .................... FAILURE [48:57 min]
...
org.apache.seatunnel.api.table.factory.FactoryException: ErrorCode:[API-06], ErrorDescription:[Factory initialize failed] - Unable to create a source for identifier 'FakeSource'.
Caused by: java.lang.NoClassDefFoundError: org/apache/seatunnel/connectors/cdc/base/source/BaseChangeStreamTableSourceFactory
Caused by: java.lang.ClassNotFoundException: org.apache.seatunnel.connectors.cdc.base.source.BaseChangeStreamTableSourceFactory
```
This is not confined to the new `MysqlCDCClusterFailoverIT` — it takes down essentially every pre-existing test in `connector-seatunnel-e2e-base` that submits a plain `FakeSource` job (`PendingJobsRestIT`, `ClusterFaultToleranceIT`, `RestApiIT`, `SeaTunnelSlotIT`, `SplitClusterFaultToleranceIT`, `SplitClusterPendingJobLifecycleFailoverIT`, `TextHeaderIT`, `SlotRatioAllocateStrategyIT`, `SystemLoadAllocateStrategyIT`, `TimerFlushIT`), 66 errors in shard 11 alone. `FakeSource` has no legitimate runtime dependency on `connector-cdc-base`'s `BaseChangeStreamTableSourceFactory`, so this reads as a classpath/shading regression in the module, not a test-logic problem in the new file.
Given this PR's own diff adds `connector-cdc-mysql` (regular + test-jar) as a new test-scope dependency to this module's `pom.xml` so the new test can reuse `MySqlContainer`/`MySqlVersion`, that's the most likely source of the break — e.g. a shaded/relocated class from `connector-cdc-base` ending up missing or in conflict once `connector-cdc-mysql`'s own dependency graph is pulled into this module's test classpath. I have not proven the exact mechanism, but the "unrelated pre-existing FakeSource tests break module-wide" signature points at the dependency addition, not at test logic in `MysqlCDCClusterFailoverIT` itself.
This needs to be root-caused and fixed before merge — it is a real, PR-caused break of unrelated existing coverage, not something to rerun past. I'd suggest checking whether the new test-jar dependency can be scoped more narrowly (e.g. excluding transitive relocations that collide with what this module already provides) or whether a `connector-cdc-mysql` test-jar is even necessary versus duplicating the small pieces of `MySqlContainer`/`MySqlVersion` needed here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Test][E2E] Add Zeta CDC cluster failover coverage [seatunnel]
Posted by "DanielLeens (via GitHub) via commits" <co...@seatunnel.apache.org>.
DanielLeens commented on PR #11947:
URL: https://github.com/apache/seatunnel/pull/11947#issuecomment-6538191678
Synced `dev` (3ac2d1cec6a) into this branch as 3b5dd6a08bd to pick up the current CI matrix and the merged upstream fixes. Plain merge commit, no conflicts; the diff against `dev` is still the same six files (`MysqlCDCClusterFailoverIT`, its resources and the `connector-seatunnel-e2e-base` pom additions this PR already carried). Compile and tests are left to the fork Build run per the validation policy: https://github.com/DanielLeens/seatunnel/actions/runs/41162877118
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org