You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "dongwooooooo (via GitHub)" <gi...@apache.org> on 2026/01/02 15:45:44 UTC
Re: [I] [Feature][Connector-v2][MySQL] Support dynamic discovery of new tables for regex-based multi-table sync [seatunnel]
dongwooooooo commented on issue #10203:
URL: https://github.com/apache/seatunnel/issues/10203#issuecomment-3705617840
Thanks @yzeng1618 for the Flink CDC reference.
Flink CDC's restore-time discovery requires a job restart, which doesn't meet this issue's "without restarting the job" requirement.
I also considered the Kafka/Pulsar pattern (background thread directly modifying state), but this could cause race conditions during checkpoints or split assignments.
I'm proposing asynchronous discovery with a pending queue to ensure thread safety: discovery thread only queues discovered tables(no state changes), and the main thread processes the queue at key points(assignSplits, snapshotState, etc) to ensure thread safety.
What do you think about this direction?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org