[jira] [Commented] (KAFKA-8178) KafkaProducer#send(ProducerRecord,Callback) may block for up to 60 seconds

["Jan Doms (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-8178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17522184#comment-17522184 ] 

Jan Doms commented on KAFKA-8178:
---------------------------------

I understand people programming in a reactive style not wanting their threads to be blocked by kafka, OTOH backpressure is important and often desirable (e.g. to limit the amount of memory occupied by a process).

As such I actually consider this documented(!) behavior a feature, which users may depend on.

See e.g. [https://kafka.apache.org/23/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html] and [https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#producerconfigs_max.block.ms] .

> The send is asynchronous and this method will return immediately once the record has been stored in the buffer of records waiting to be sent.

I want to explicitly ask to not change this behavior for existing users. If the changed behavior is to be implemented it should probably be done as a new overload of the send method allowing users to opt in to it.

> KafkaProducer#send(ProducerRecord,Callback) may block for up to 60 seconds
> --------------------------------------------------------------------------
>
>                 Key: KAFKA-8178
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8178
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients, producer 
>            Reporter: Sergei Egorov
>            Priority: Major
>
> Hello. I was running reactor-kafka with [the BlockHound agent|https://github.com/reactor/BlockHound] (you can see the progress [here|https://github.com/reactor/reactor-kafka/pull/75] and even run it yourself) and it detected a very dangerous blocking call in KafkaProducer#send(ProducerRecord,Callback) which is supposed to be async:
> {code:java}
> java.lang.Error: Blocking call! java.lang.Object#wait
> 	at reactor.BlockHound$Builder.lambda$new$0(BlockHound.java:154)
> 	at reactor.BlockHound$Builder.lambda$install$8(BlockHound.java:254)
> 	at reactor.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:43)
> 	at java.lang.Object.wait(Object.java)
> 	at org.apache.kafka.clients.Metadata.awaitUpdate(Metadata.java:181)
> 	at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:938)
> 	at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:823)
> 	at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:803)
> {code}
> it blocks for up to "maxBlockTimeMs" (60 seconds by default)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)