[jira] [Commented] (KAFKA-20458) AcknowledgeType.RENEW re-delivers records to application via poll() in explicit acknowledgement mode, undocumented and inconsistent with broker-side contract

["Andrew Schofield (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-20458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18074925#comment-18074925 ] 

Andrew Schofield commented on KAFKA-20458:
------------------------------------------

The javadoc for `KafkaShareConsumer` explains the behaviour for renew acknowledgements. It's Option B above.

> AcknowledgeType.RENEW re-delivers records to application via poll() in explicit acknowledgement mode, undocumented and inconsistent with broker-side contract
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-20458
>                 URL: https://issues.apache.org/jira/browse/KAFKA-20458
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 4.2.0
>            Reporter: Shekhar Prasad Rajak
>            Assignee: Shekhar Prasad Rajak
>            Priority: Major
>
> AcknowledgeType.RENEW is documented as a way to extend the acquisition lock on a record that is still being processed, without changing the record's state on the broker. However, the client-side implementation silently re-delivers the same records back to the application on the next poll() call. This behavior is nowhere documented, violates the explicit acknowledgement mode invariant, and diverges from the broker-side contract.
> RENEW simultaneously satisfies two contradictory contracts:
> Context ----What RENEW implies
> ensureInFlightAcknowledgedIfExplicitAcknowledgement() --- Record IS acknowledged — gate passes
> collect() / takeRenewedRecords() ----Record is NOT done — re-delivered on next poll()
> Two acceptable resolutions, depending on intended design:
> Option A — RENEW means "extend lock only, no re-delivery":
> Option B — RENEW means "extend lock and re-deliver to same consumer" (current behavior, make it explicit).
> Our usecase : 
> We are building an async pipeline where records are consumed, published to a downstream system asynchronously, and then acknowledged. We need to:
>  1.  Hold the broker acquisition lock while async publish is in flight.
>  2.  Not lose the record if publish takes longer than one poll cycle.
>  3.  Call ACCEPT once publish confirms, RELEASE if it fails.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)