You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "ShivsundarR (via GitHub)" <gi...@apache.org> on 2024/11/15 12:12:15 UTC

[PR] KAFKA-18016: Modified handling of piggyback acknowledgements in ShareConsumeRequestManager. [kafka]

ShivsundarR opened a new pull request, #17824:
URL: https://github.com/apache/kafka/pull/17824

   *What*
   There was a bug in handling piggyback acknowledgements in `ShareConsumeRequestManager`, where the `fetchAcknowledgementsMap` could be updated when the request was in flight and when the `ShareFetch` response is received, we were removing any acknowledgements(without actually sending them) which came when the request was in flight.
   
   *Fix*
   Now we are maintaining 2 separate maps(one which has the acknowledgements to send and one which keeps track of the acknowledgements in flight).
   
   *Testing*
   Added a unit test which passes after this fix.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18016: Modified handling of piggyback acknowledgements in ShareConsumeRequestManager. [kafka]

Posted by "ShivsundarR (via GitHub)" <gi...@apache.org>.
ShivsundarR commented on code in PR #17824:
URL: https://github.com/apache/kafka/pull/17824#discussion_r1844241413


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumeRequestManager.java:
##########
@@ -566,8 +573,11 @@ public CompletableFuture<Void> acknowledgeOnClose(final Map<TopicIdPartition, Ac
                 for (TopicIdPartition tip : sessionHandler.sessionPartitions()) {
                     Acknowledgements acknowledgements = acknowledgementsMap.getOrDefault(tip, Acknowledgements.empty());
 
-                    if (fetchAcknowledgementsMap.get(tip) != null) {
-                        acknowledgements.merge(fetchAcknowledgementsMap.remove(tip));
+                    Acknowledgements acksFromShareFetch = fetchAcknowledgementsToSend.remove(tip);
+
+                    if (acksFromShareFetch != null) {
+                        acknowledgements.merge(acksFromShareFetch);
+                        fetchAcknowledgementsInFlight.put(tip, acksFromShareFetch);

Review Comment:
   Yeah right makes sense, these acks would just remain over in the inFlight map. I have now removed the line and updated the PR. Thanks.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18016: Modified handling of piggyback acknowledgements in ShareConsumeRequestManager. [kafka]

Posted by "omkreddy (via GitHub)" <gi...@apache.org>.
omkreddy merged PR #17824:
URL: https://github.com/apache/kafka/pull/17824


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] KAFKA-18016: Modified handling of piggyback acknowledgements in ShareConsumeRequestManager. [kafka]

Posted by "AndrewJSchofield (via GitHub)" <gi...@apache.org>.
AndrewJSchofield commented on code in PR #17824:
URL: https://github.com/apache/kafka/pull/17824#discussion_r1844107332


##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumeRequestManager.java:
##########
@@ -566,8 +573,11 @@ public CompletableFuture<Void> acknowledgeOnClose(final Map<TopicIdPartition, Ac
                 for (TopicIdPartition tip : sessionHandler.sessionPartitions()) {
                     Acknowledgements acknowledgements = acknowledgementsMap.getOrDefault(tip, Acknowledgements.empty());
 
-                    if (fetchAcknowledgementsMap.get(tip) != null) {
-                        acknowledgements.merge(fetchAcknowledgementsMap.remove(tip));
+                    Acknowledgements acksFromShareFetch = fetchAcknowledgementsToSend.remove(tip);
+
+                    if (acksFromShareFetch != null) {
+                        acknowledgements.merge(acksFromShareFetch);
+                        fetchAcknowledgementsInFlight.put(tip, acksFromShareFetch);

Review Comment:
   It doesn't seem necessary to put the acknowledgements into the `fetchAcknowledgementsInFlight` map in this case. We are closing the share session using a `ShareAcknowledge` and this code is just picking up any acknowledgements which were intended to be piggybacked on the next fetch. They have essentially been taken over by this logic and don't need to be accounted for as fetch acknowledgments in flight.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org