[jira] [Created] (KAFKA-20458) AcknowledgeType.RENEW re-delivers records to application via poll() in explicit acknowledgement mode, undocumented and inconsistent with broker-side contract

["Shekhar Prasad Rajak (Jira)" <ji...@apache.org>]

Shekhar Prasad Rajak created KAFKA-20458:
--------------------------------------------

             Summary: AcknowledgeType.RENEW re-delivers records to application via poll() in explicit acknowledgement mode, undocumented and inconsistent with broker-side contract
                 Key: KAFKA-20458
                 URL: https://issues.apache.org/jira/browse/KAFKA-20458
             Project: Kafka
          Issue Type: Bug
    Affects Versions: 4.2.0
            Reporter: Shekhar Prasad Rajak


AcknowledgeType.RENEW is documented as a way to extend the acquisition lock on a record that is still being processed, without changing the record's state on the broker. However, the client-side implementation silently re-delivers the same records back to the application on the next poll() call. This behavior is nowhere documented, violates the explicit acknowledgement mode invariant, and diverges from the broker-side contract.

RENEW simultaneously satisfies two contradictory contracts:

Context ----What RENEW implies
ensureInFlightAcknowledgedIfExplicitAcknowledgement() --- Record IS acknowledged — gate passes
collect() / takeRenewedRecords() ----Record is NOT done — re-delivered on next poll()


Two acceptable resolutions, depending on intended design:
Option A — RENEW means "extend lock only, no re-delivery":

Option B — RENEW means "extend lock and re-deliver to same consumer" (current behavior, make it explicit).

Our usecase : 

We are building an async pipeline where records are consumed, published to a downstream system asynchronously, and then acknowledged. We need to:

 1.  Hold the broker acquisition lock while async publish is in flight.
 2.  Not lose the record if publish takes longer than one poll cycle.
 3.  Call ACCEPT once publish confirms, RELEASE if it fails.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)