You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "201811510411lw (via GitHub)" <gi...@apache.org> on 2026/08/26 10:01:20 UTC
[I] [Bug][Connector-V2][MySQL-CDC] snapshotSplitColumn rejects a physical primary key [seatunnel]
201811510411lw opened a new issue, #11973:
URL: https://github.com/apache/seatunnel/issues/11973
### Search before asking
- [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues.
### What happened
When `snapshotSplitColumn` is explicitly configured with a physical primary key,
SeaTunnel rejects it if the column is not returned by
`JdbcDataSourceDialect.getUniqueKeys()`.
Example table:
```sql
CREATE TABLE test_split_column (
id BIGINT NOT NULL AUTO_INCREMENT,
category TINYINT NOT NULL,
business_key VARCHAR(64) NOT NULL,
PRIMARY KEY (id),
UNIQUE KEY uk_business (category, business_key)
);
### SeaTunnel Version
2.3.13-SNAPSHOT
The same issue is present on the current dev branch at commit:
1491a8e0ad6f0f77ed4853fd4db3d4ff1cdfbd2e
### SeaTunnel Config
```conf
source {
MySQL-CDC {
url = "jdbc:mysql://<host>:3306/test"
username = "******"
password = "******"
database-names = ["test"]
table-names = ["test.test_split_column"]
table-names-config = [{
table = "test.test_split_column"
primaryKeys = ["id"]
snapshotSplitColumn = "id"
}]
startup.mode = "initial"
snapshot.split.size = 4096
snapshot.fetch.size = 512
}
}
```
### Running Command
```shell
# The job was submitted through SeaTunnel on Flink
# Flink version: 1.18.1
# Equivalent command for a local reproduction:
./bin/start-seatunnel-flink-15-connector-v2.sh \
--config ./config/mysql-cdc-primary-key.conf
```
### Error Exception
```log
Config snapshotSplitColumn not unique key for table test.test_split_column
Splitting table test.test_split_column into chunks,
split column: category, min: 1, max: 2, chunk size: 4096
```
### Zeta or Flink or Spark Version
_No response_
### Java or Scala Version
_No response_
### Screenshots
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][Connector-V2][MySQL-CDC] snapshotSplitColumn rejects a physical primary key [seatunnel]
Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on issue #11973:
URL: https://github.com/apache/seatunnel/issues/11973#issuecomment-5438933393
Thanks for the detailed downstream verification, and thanks again to @waterWang and @zhangshenghang for the quick follow-up PRs.
I rechecked the current `apache/dev` head (`e6caf5a4b337`). The issue is still grounded there: `AbstractJdbcSourceChunkSplitter.getSplitColumn()` first validates an explicit `snapshotSplitColumn` only against `dialect.getUniqueKeys()`, and only later falls back to primary-key / unique-key discovery for automatic split-column selection. That means an explicitly configured physical primary key can still be rejected before the automatic fallback path is reached.
Your August 27, 2026 verification result is useful because it shows the intended behavior directly:
- the configured physical primary key was accepted;
- the previous `Config snapshotSplitColumn not unique key` path disappeared;
- the generated snapshot splits used the configured primary-key boundaries.
At this point, the strongest current repair signal is #11979: it has your downstream validation and its current Build is green. #11977 is still open but its current Build is failing. Please keep the fix path converged on one implementation and one focused regression test, rather than landing two near-duplicate repairs.
For review, the contract should stay narrow:
1. accept an explicitly configured split column when it belongs to the physical primary key or a unique key;
2. preserve the existing automatic fallback behavior when the config is absent or invalid;
3. keep the regression coverage focused on this exact configured-primary-key path.
Unless the same scenario still reproduces on current `dev` with the #11979 logic, the next maintainer action belongs on PR review rather than more issue-thread debugging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][Connector-V2][MySQL-CDC] snapshotSplitColumn rejects a physical primary key [seatunnel]
Posted by "201811510411lw (via GitHub)" <gi...@apache.org>.
201811510411lw commented on issue #11973:
URL: https://github.com/apache/seatunnel/issues/11973#issuecomment-5434896342
Thanks @waterWang and @zhangshenghang for investigating this issue and providing fixes.
- @waterWang proposed #11977.
- @zhangshenghang proposed #11979.
I verified the core fix from #11979 after cherry-picking it into our downstream branch based on SeaTunnel 2.3.13.
The test configuration included:
```hocon
table-names-config = [{
table = "<redacted_table>"
primaryKeys = ["ID"]
snapshotSplitColumn = "ID"
}]
startup.mode = "initial"
snapshot.split.size = 4096
```
The configured physical primary key was accepted as the snapshot split column:
```text
Chosen split column ID for table <redacted_table>
Splitting table <redacted_table> into chunks,
split column: ID,
min: 1,
max: 1280590,
chunk size: 4096
Split table <redacted_table> into 305 chunks.
```
The generated snapshot splits used non-null `ID` boundaries. For example:
```text
SnapshotSplit(
tableId=<redacted_table>,
splitKeyType=ROW<ID BIGINT>,
splitStart=[1223656],
splitEnd=[1227861]
)
```
The previous behavior no longer occurred:
```text
Config snapshotSplitColumn not unique key
```
SeaTunnel did not fall back to another unique-key column.
Therefore, the fix from #11979 resolves the reported issue in our SeaTunnel 2.3.13-based runtime.
Thank you both again for the quick response and contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug][Connector-V2][MySQL-CDC] snapshotSplitColumn rejects a physical primary key [seatunnel]
Posted by "nzw921rx (via GitHub)" <gi...@apache.org>.
nzw921rx closed issue #11973: [Bug][Connector-V2][MySQL-CDC] snapshotSplitColumn rejects a physical primary key
URL: https://github.com/apache/seatunnel/issues/11973
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org