You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "DanielLeens (via GitHub)" <gi...@apache.org> on 2026/06/28 01:46:41 UTC

[PR] [Fix][E2E] Isolate Postgres CDC replication slots per test run [seatunnel]

DanielLeens opened a new pull request, #11202:
URL: https://github.com/apache/seatunnel/pull/11202

   ## What does this PR do?
   
   This fixes a shared Postgres CDC E2E instability that is unrelated to JDBC catalog compatibility changes in #11169.
   
   The root cause is that `PostgresCDCIT` reuses the default replication slot name (`seatunnel`) across multiple streaming jobs and restore flows. When CI runs overlap, one test can fail with `replication slot "seatunnel" already exists`, which then cascades into unrelated PR failures.
   
   ## How was this fixed?
   
   - assign a unique `slot.name` for each `PostgresCDCIT` job invocation
   - reuse the same generated slot across savepoint/restore flows in the same test
   - isolate the Debezium JSON Kafka consumer group with the same per-test slot identity
   - wire the relevant Postgres CDC E2E config files to accept `slot_name` injection
   
   ## Verification
   
   - `./mvnw spotless:apply`
   - `./mvnw -pl seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-postgres-e2e -DskipUT=true -DskipIT=false -Dit.test=PostgresCDCIT -DfailIfNoTests=false -Dskip.spotless=true -o verify`
   
   The module-level E2E run reached `PostgresCDCIT`, started Docker/Testcontainers successfully after bringing Docker Desktop up, and then was blocked by the local Docker daemon network pool state: `all predefined address pools have been fully subnetted`. This is a local Docker environment limitation during validation, not a test assertion failure from the code change itself.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][E2E] Wait for Postgres CDC WAL stream readiness [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo merged PR #11202:
URL: https://github.com/apache/seatunnel/pull/11202


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][E2E] Wait for Postgres CDC WAL stream readiness [seatunnel]

Posted by "heye1005 (via GitHub)" <gi...@apache.org>.
heye1005 commented on code in PR #11202:
URL: https://github.com/apache/seatunnel/pull/11202#discussion_r3498137111


##########
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-postgres-e2e/src/test/java/org/apache/seatunnel/connectors/seatunnel/cdc/postgres/PostgresCDCIT.java:
##########
@@ -251,6 +254,28 @@ private Properties kafkaConsumerConfig() {
         return props;
     }
 
+    /**
+     * Replication slots are shared inside the reused Postgres test container, so each CDC job needs
+     * an isolated slot to avoid cross-test collisions when streaming jobs overlap.
+     */
+    private String createSlotName() {
+        return "seatunnel_" + Long.toHexString(JobIdGenerator.newJobId());

Review Comment:
   use GENERATED_SLOT_PREFIX



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][E2E] Wait for Postgres CDC WAL stream readiness [seatunnel]

Posted by "heye1005 (via GitHub)" <gi...@apache.org>.
heye1005 commented on code in PR #11202:
URL: https://github.com/apache/seatunnel/pull/11202#discussion_r3498176674


##########
seatunnel-e2e/seatunnel-connector-v2-e2e/connector-cdc-postgres-e2e/src/test/java/org/apache/seatunnel/connectors/seatunnel/cdc/postgres/PostgresCDCIT.java:
##########
@@ -311,29 +339,40 @@ public void testPostgresCdcWithDebeziumJsonFormat(TestContainer container) {
                                 dataSize.updateAndGet(v -> v + getKafkaData().size());
                                 Assertions.assertEquals(1, dataSize.get());
                             });
-            // insert update delete
-            upsertDeleteSourceTable(POSTGRESQL_SCHEMA, SOURCE_TABLE_NO_PRIMARY_KEY_DEBEZIUM);
-
-            await().atMost(1000 * 60 * 3, TimeUnit.MILLISECONDS)

Review Comment:
   before 3min,after 1min, may cause flaky problems



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org