[PR] KAFKA-21181: Allow committing an empty transaction in TV2 (4.0) [kafka]

["rreddy-22 (via GitHub)" <gi...@apache.org>]

rreddy-22 opened a new pull request, #23887:
URL: https://github.com/apache/kafka/pull/23887

   Backport of #23831 to 4.0 (cherry-pick of 814cfa34ebd4b13e09b5eef7782f99bd82f1e9ea).
   
   With transaction V2, an EndTxn(COMMIT) at the current epoch on a transaction in Empty, CompleteCommit or CompleteAbort state returned a fatal `INVALID_TXN_STATE`. A coordinator that has lost leadership but not yet applied the metadata change sees the transaction in exactly this state, since its partitions were added on the new coordinator. This commits an empty transaction instead, the same way TV2 already handles an empty abort: bump the epoch, write PrepareCommit with no partitions and complete it without markers. A stale coordinator cannot replicate that `acks=-1` write, so the client gets a retriable error and commits on the new coordinator.
   
   **Backport notes:** 4.0 does not have KAFKA-20785, so CompleteAbort + Commit + Retry still returns `INVALID_TXN_STATE` there rather than `PRODUCER_FENCED` (the upstream commit message's "still returns PRODUCER_FENCED" does not apply). That retry behavior is kept; only the current-epoch commit becomes an empty commit:
   - `TransactionCoordinator`: the CompleteAbort row of the state table is `NONE | EB | ITS | EB`, the KAFKA-20785 note is not included, and the CompleteAbort + Commit branch returns ITS on retry and calls `commitEmptyTransaction()` otherwise.
   - `TransactionCoordinatorTest`: kept 4.0's TV0/TV2 parameterized `shouldReturnInvalidTxnRequestOnEndTxnRequestWhenStatusIsCompleteAbortAndResultIsNotAbort`, added `shouldCommitEmptyTransactionOnEndTxnRequestWhenStatusIsCompleteAbortAndResultIsCommitInV2`, and left out the two KAFKA-20785 `PRODUCER_FENCED` tests.
   - `TransactionMetadata` and the `TransactionState` objects are Scala in `core` on 4.0, so the changes were ported to `TransactionMetadata.scala`: `PrepareCommit`'s valid previous states now include `Empty`, `CompleteCommit` and `CompleteAbort`, its scaladoc and `Empty`'s were updated, and `allowedEmptyAbort` became `allowedEmptyCompletion`. `testTransitFromNoOngoingTransactionToPrepareCommitAndCompleteCommitInV2` was ported to the Scala `TransactionMetadataTest`.
   - The new tests use 4.0's Scala `TransactionState` objects (`@ValueSource(strings)` with `TransactionState.fromName` instead of `@EnumSource`) and Scala collections.
   - `gradle/spotbugs-exclude.xml` gets one narrow exclusion (`EC_UNRELATED_TYPES` in `TransactionCoordinator.commitEmptyTransaction$1`). After scalac inlines `generateTxnTransitMetadataForTxnCompletion` into `commitEmptyTransaction`, its `nextState == PrepareAbort` check compares the `PrepareCommit` and `PrepareAbort` case objects, which spotBugs flags. The comparison is intended to be false. This only affects 4.0, where the states are Scala objects rather than a Java enum, and it follows the existing scalac-inlining exclusion for `kafka.log.Log`.
   
   **Testing:** `./gradlew :core:test --tests 'kafka.coordinator.transaction.*' :core:checkstyleTest :core:checkstyleMain` and `:core:spotbugsMain` pass on 4.0, including all new and renamed tests.
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   
   Reviewers: Justine Olshan <jo...@confluent.io>
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] KAFKA-21181: Allow committing an empty transaction in TV2 (4.0) [kafka]

["jolshan (via GitHub)" <gi...@apache.org>]

jolshan merged PR #23887:
URL: https://github.com/apache/kafka/pull/23887


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org