[GitHub] [kafka] hachikuji commented on a change in pull request #8239: KAFKA-9666: Don't increase transactional epoch when trying to fence if the log append fails

[GitBox <gi...@apache.org>]

hachikuji commented on a change in pull request #8239:
URL: https://github.com/apache/kafka/pull/8239#discussion_r420974886



##########
File path: core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala
##########
@@ -487,6 +487,33 @@ class TransactionCoordinator(brokerId: Int,
               info(s"Aborting sending of transaction markers and returning $error error to client for $transactionalId's EndTransaction request of $txnMarkerResult, " +
                 s"since appending $newMetadata to transaction log with coordinator epoch $coordinatorEpoch failed")
 
+              txnManager.getTransactionState(transactionalId).right.foreach {

Review comment:
       Took me a while to remember this issue... So basically the coordinator has decided to abort a transaction and has bumped the epoch. However, when it tries to write the updated state to the log, it fails, which leaves us in an inconsistent state. Of course failing to write to the log doesn't necessarily mean that the entry wasn't appended. In fact, it could still become committed. There is no way to take the write back once it gets to the log. Hence I'm a little hesitant about the logic to revert to the previous epoch in this case. Would it still be possible for the fenced producer to make progress with the old epoch after reverting? Perhaps another idea would be to keep the epoch bumped in memory, but remember the fact that the write had failed. So the next time we go to retry the abort, we do not need to bump the epoch again. Does that make sense?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org