You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jira@kafka.apache.org by "splett2 (via GitHub)" <gi...@apache.org> on 2024/03/20 18:41:23 UTC
[PR] KAFKA-16395: Producer should refresh metadata when a request is cancelled due to request timeout [kafka]
splett2 opened a new pull request, #15565:
URL: https://github.com/apache/kafka/pull/15565
### What
On a client-side triggered request timeout (the client did not receive a response from the server in a timely manner), the client needs to refresh metadata. This requires the partition response passed to `completeBatch` to be an `instanceof InvalidMetadataException`.
This is slightly different than Errors.REQUEST_TIMED_OUT, since the latter is returned when the HWM fails to advance within the configured request timeout threshold, so we do not want to make `REQUEST_TIMED_OUT` an invalid metadata exception.
I couldn't find a better fit than NETWORK_EXCEPTION, so hopefully the improved logging from KAFKA-14317 still reasonably distinguishes between request timeouts and sudden connection interrupts.
### Testing
Added a unit test in `SenderTest` using
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16395: Producer should refresh metadata when a request is cancelled due to request timeout [kafka]
Posted by "splett2 (via GitHub)" <gi...@apache.org>.
splett2 commented on PR #15565:
URL: https://github.com/apache/kafka/pull/15565#issuecomment-2010669973
I took a closer look and I am pretty sure that my repro is incorrect. I think the network client was behaving correctly based on my reading of the code from KAFKA-14317, but perhaps it wasn't logging out when metadata refreshes were being requested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16395: Producer should refresh metadata when a request is cancelled due to request timeout [kafka]
Posted by "splett2 (via GitHub)" <gi...@apache.org>.
splett2 closed pull request #15565: KAFKA-16395: Producer should refresh metadata when a request is cancelled due to request timeout
URL: https://github.com/apache/kafka/pull/15565
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] KAFKA-16395: Producer should refresh metadata when a request is cancelled due to request timeout [kafka]
Posted by "ijuma (via GitHub)" <gi...@apache.org>.
ijuma commented on code in PR #15565:
URL: https://github.com/apache/kafka/pull/15565#discussion_r1532671434
##########
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java:
##########
@@ -586,7 +586,7 @@ private void handleProduceResponse(ClientResponse response, Map<TopicPartition,
log.trace("Cancelled request with header {} due to the last request to node {} timed out",
requestHeader, response.destination());
for (ProducerBatch batch : batches.values())
- completeBatch(batch, new ProduceResponse.PartitionResponse(Errors.REQUEST_TIMED_OUT, String.format("Disconnected from node %s due to timeout", response.destination())),
+ completeBatch(batch, new ProduceResponse.PartitionResponse(Errors.NETWORK_EXCEPTION, String.format("Disconnected from node %s due to request timeout", response.destination())),
Review Comment:
Please add a comment clarifying that this cannot be REQUEST_TIMEOUT even though the message says so
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscribe@kafka.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org