[jira] [Commented] (KAFKA-7412) Bug prone response from producer.send(ProducerRecord, Callback) if Kafka broker is not running

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-7412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649959#comment-16649959 ] 

ASF GitHub Bot commented on KAFKA-7412:
---------------------------------------

huxihx opened a new pull request #5798: KAFKA-7412: onComplete should not reassign `metadata` variable
URL: https://github.com/apache/kafka/pull/5798
 
 
   The Java doc for `InterceptorCallback#onComplete` says that exactly one of the arguments(metadata and exception) will be non-null. However, the commitment will be broken when TimeoutException is encountered since the code reassigns a new-created RecordMetadata object to variable `metadata`.
   
   The solution is to leave `metadata1` unchanged and pass a new RecordMetadata instance to `onAcknowledgement`.
   
   *More detailed description of your change,
   if necessary. The PR title and PR message become
   the squashed commit message, so use a separate
   comment to ping reviewers.*
   
   *Summary of testing strategy (including rationale)
   for the feature or bug fix. Unit and/or integration
   tests are expected for any behaviour change and
   system tests should be considered for larger changes.*
   
   ### Committer Checklist (excluded from commit message)
   - [ ] Verify design and implementation 
   - [ ] Verify test coverage and CI build status
   - [ ] Verify documentation (including upgrade notes)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Bug prone response from producer.send(ProducerRecord, Callback) if Kafka broker is not running
> ----------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-7412
>                 URL: https://issues.apache.org/jira/browse/KAFKA-7412
>             Project: Kafka
>          Issue Type: Bug
>          Components: producer 
>    Affects Versions: 2.0.0
>            Reporter: Michal Turek
>            Priority: Major
>         Attachments: both_metadata_and_exception.png, metadata_when_kafka_is_stopped.png
>
>
> Hi there, I have probably found a bug in Java Kafka producer client.
> Scenario & current behavior:
> - Start Kafka broker, single instance.
> - Start application that produces messages to Kafka.
> - Let the application to load partitions for a topic to warm up the producer, e.g. send a message to Kafka. I'm not sure if this is necessary step, but our code does it.
> - Gracefully stop the Kafka broker.
> - Application logs now contains "org.apache.kafka.clients.NetworkClient: [Producer clientId=...] Connection to node 0 could not be established. Broker may not be available." so the client is aware about the Kafka unavailability.
> - Trigger the producer to send a message using KafkaProducer.send(ProducerRecord, Callback) method.
> - The callback that notifies business code receives non-null RecordMetadata and null Exception after request.timeout.ms. The metadata contains offset -1 which is value of ProduceResponse.INVALID_OFFSET.
> Expected behavior:
> - If the Kafka is not running and the message is not appended to the log, the callback should contain null RecordMetadata and non-null Exception. At least I subjectively understand the Javadoc this way, "exception on production error" in simple words.
> - Developer that is not aware of this behavior and that doesn't test for offset -1, may consider the message as successfully send and properly acked by the broker.
> Known workaround
> - Together with checking for non-null exception in the callback, add another condition for ProduceResponse.INVALID_OFFSET.
> {noformat}
>                 try {
>                     producer.send(record, (metadata, exception) -> {
>                         if (metadata != null) {
>                             if (metadata.offset() != ProduceResponse.INVALID_OFFSET) {
>                                 // Success
>                             } else {
>                                 // Failure
>                             }
>                         } else {
>                             // Failure
>                         }
>                     });
>                 } catch (Exception e) {
>                     // Failure
>                 }
> {noformat}
> Used setup
> - Latest Kafka 2.0.0 for both broker and Java client.
> - Originally found with broker 0.11.0.1 and client 2.0.0.
> - Code is analogy of the one in Javadoc of KafkaProducer.send().
> - Used producer configuration (others use defaults).
> {noformat}
>     bootstrap.servers = "localhost:9092"
>     client.id = "..."
>     acks = "all"
>     retries = 1
>     linger.ms = "20"
>     compression.type = "lz4"
>     request.timeout.ms = 5000 # The same behavior is with default, this is to speed up the tests
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)