You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "JacobZheng0927 (via GitHub)" <gi...@apache.org> on 2026/08/20 03:29:18 UTC

[PR] [Fix][Connector-V2][Jdbc] Prevent sink batch replay after flush failure [seatunnel]

JacobZheng0927 opened a new pull request, #11890:
URL: https://github.com/apache/seatunnel/pull/11890

   ### Purpose of this pull request
   
   A terminal JDBC sink flush failure could leave buffered records pending. A later `flush()` or `close()` could execute the same batch again, risking duplicate writes.
   
   This patch latches terminal flush and batch-clear failures, prevents subsequent flush attempts after that terminal state, and makes buffered executors retain their buffers until a successful delegate execution.
   
   ### Does this PR introduce _any_ user-facing change?
   
   Yes, as a bug fix.
   
   Previously, a failed sink flush could be followed by another execution of the same batch during a later flush or writer close. Now the original failure is preserved and subsequent flushes fail without replaying the batch.
   
   ### How was this patch tested?
   
   Added regression coverage for retry exhaustion, executor failures, `clearBatch()` failure after row-level handling, and repeated `flush()` / `close()` calls after terminal failure. The tests assert that the executor is invoked only once after such a failure.
   
   Ran focused connector-JDBC unit tests locally with JDK 8:
   
   ```shell
   ./mvnw -pl seatunnel-connectors-v2/connector-jdbc -am \
     -Dtest=JdbcOutputFormatTest,InsertOrUpdateBatchStatementExecutorTest \
     -DfailIfNoTests=false test
   ```
   
   Result: 8 tests run, 0 failures, 0 errors.
   
   ### Check list
   
   * [x] No new Jar binary package is added.
   * [x] Documentation and incompatible-change updates are not required for this bug fix.
   * [x] No new connector, plugin mapping, distribution dependency, or connector configuration is required.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2][Jdbc] Prevent sink batch replay after flush failure [seatunnel]

Posted by "DanielLeens (via GitHub) via commits" <co...@seatunnel.apache.org>.
DanielLeens commented on PR #11890:
URL: https://github.com/apache/seatunnel/pull/11890#issuecomment-6538130300

   The Build for the current c8fde87606b926e3eb7b5fcfeba4763b649a8f98 head has completed successfully. This supersedes the pending check in my latest full review; the earlier green result on 298186e was not the evidence for this revision. My source conclusion remains clear for retained terminal flush failures and close-without-replay behavior, so the CI objection is withdrawn and this can proceed to final maintainer review.
   
   Checked 2026-10-31T03:00:29.395797+00:00; unchanged head `c8fde87606b926e3eb7b5fcfeba4763b649a8f98`. [Signal pull request update](https://github.com/apache/seatunnel/actions/runs/40990602639/job/123852176155)=`SUCCESS`; [Build](https://github.com/apache/seatunnel/runs/123861660085)=`SUCCESS`.
   
   This is a discussion/CI clarification, not another full source review or an effective maintainer approval. Daniel has READ repository permission; a write-capable maintainer owns required approval/dismissal.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2][Jdbc] Prevent sink batch replay after flush failure [seatunnel]

Posted by "davidzollo (via GitHub)" <gi...@apache.org>.
davidzollo commented on PR #11890:
URL: https://github.com/apache/seatunnel/pull/11890#issuecomment-6393847039

   Resolved the current branch conflicts against `dev` (`d676f91b3d6aaa2cca693816697a211e4d384805`) in `25f4d8cdd3998721aa20101c3fbe132eea93cf94`. The resolution preserves current checkpoint/manual-commit boundaries, row-error rollback/reset, commit-failure handling, and connection-recovery behavior. JDBC batch execution remains owned by `JdbcOutputFormat`; executor resource closing does not replay a failed batch. Fatal connector exceptions now set the current upstream failure latch as well.
   
   The existing upstream clear-batch regression test is preserved alongside the no-replay tests. Scoped Spotless and the target-relative whitespace check passed; independent maintainer source/integration review passed.
   
   The old failed-job log confirms the stale `JobStateEventTest` undefined `FAILED_JOB_EVENT_TIMEOUT_SECONDS` compile error, already fixed upstream by #11954; this sync includes that fix. The remote new head and six-file final PR scope were read back. New-head CI and mergeability had not yet been reported at immediate readback, so runtime validation remains pending.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2][Jdbc] Prevent sink batch replay after flush failure [seatunnel]

Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11890:
URL: https://github.com/apache/seatunnel/pull/11890#issuecomment-6473562247

   Thanks for the contribution. The CI fact cited in my previous conclusion has changed; this follow-up updates that gate on the unchanged reviewed head, without issuing another full source review.
   
   Gate snapshot (2026-10-28T15:44:37+00:00), head `298186e26f44ce4a019590964354db316f414397`: [Build](https://github.com/apache/seatunnel/runs/122809644085) is `SUCCESS`.
   
   CI was previously the only remaining technical gate in my conclusion; CI is now passing, so that CI objection is withdrawn and my technical conclusion is clear to proceed. Daniel currently has READ permission. I have not changed the effective GitHub review state to APPROVED; a write-capable maintainer must perform the final approve/dismiss action required by branch protection.
   
   Comparison recorded in this scan: base `dev`, `diverged`, ahead 5 / behind 48, no merge conflict. Branch drift is metadata here and is not an additional sync request.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2][Jdbc] Prevent sink batch replay after flush failure [seatunnel]

Posted by "DanielLeens (via GitHub)" <gi...@apache.org>.
DanielLeens commented on PR #11890:
URL: https://github.com/apache/seatunnel/pull/11890#issuecomment-5459189299

   CI update on my 2026-08-24 re-review, @JacobZheng0927 — at the time I wrote "Build was still IN_PROGRESS... I have no reason to expect a failure," but that run finished shortly after (7 minutes later) with `FAILURE`, so I want to correct the record and give you the concrete cause and fix, rather than leaving "wait for CI" as the last word.
   
   Root cause: it's a real compile failure, but it is not in this PR's diff. The full log shows:
   
   ```
   [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.10.1:testCompile (default-testCompile) on project seatunnel-engine-server: Compilation failure
   [ERROR] .../seatunnel-engine/seatunnel-engine-server/src/test/java/org/apache/seatunnel/engine/server/event/JobStateEventTest.java:[165,23] error: cannot find symbol
   [ERROR]   symbol:   variable FAILED_JOB_EVENT_TIMEOUT_SECONDS
   [ERROR]   location: class JobStateEventTest
   ```
   
   That test compile failure fails the whole reactor build, which is why every single downstream job (unit tests, all the connector/CDC ITs, etc.) shows red on this run — it's a single shared-step break, not dozens of independent failures.
   
   This branch's merge commit (`caa401bb7f`, "Merge remote-tracking branch 'origin/dev'") synced with `dev` at a point where `JobStateEventTest` referenced that constant name, before it was fixed upstream. I confirmed `dev` already contains the fix: commit `43fe63b1fc` "[Fix][Zeta] Fix undefined job event timeout constant (#11954)" landed on `dev` at 2026-08-24T11:50:50Z — about 10 hours after this PR's merge-commit CI run started. Current `dev` uses `RESTORE_TO_FAILED_TIMEOUT_SECONDS` in that test, so the symbol issue is already resolved upstream.
   
   Concrete next step: sync your branch with the latest `dev` again (merge or rebase) and push. That will pick up #11954's fix and the `seatunnel-engine-server` test-compile step should pass again. This is unrelated to your JDBC sink flush fix itself — I already re-verified in my last review that the 7 files in this PR's actual diff (`JdbcOutputFormat.java`, the three executors, `JdbcSinkWriter.java`) are byte-for-byte identical to what I approved before, so no further code review is needed once CI is green again.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Fix][Connector-V2][Jdbc] Prevent sink batch replay after flush failure [seatunnel]

Posted by "JacobZheng0927 (via GitHub)" <gi...@apache.org>.
JacobZheng0927 commented on PR #11890:
URL: https://github.com/apache/seatunnel/pull/11890#issuecomment-5389744579

   Merged the current `dev` branch to resolve the conflict while preserving `outputFormat.flush()` before the non-autocommit commit path. Focused JDBC regression tests passed (11/11), and `spotless:check` passed locally with JDK 8. Updated head: `caa401bb7`.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org