[jira] [Commented] (KAFKA-1789) Issue with Async producer

["Ismael Juma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15337883#comment-15337883 ] 

Ismael Juma commented on KAFKA-1789:
------------------------------------

To add to Gwen's comment, the Scala producers have been deprecated and the new Java producer is recommended instead.

> Issue with Async producer
> -------------------------
>
>                 Key: KAFKA-1789
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1789
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8.0, 0.8.1
>            Reporter: devendra tagare
>            Priority: Critical
>
> Hi,
> We are using an async producer to send data to a kafka cluster.The event rate at peak is around 250 events/second of size 25KB each.
> In the producer code base we have added specific debug statements to capture the time taken to create a producer,create a keyed message with a byte payload & send the message.
> We have added the below properties to the producerConfig
> queue.enqueue.timeout.ms=20
> send.buffer.bytes=1024000
> topic.metadata.refresh.interval.ms=30000
> Based on the documentation, producer.send() queues the message on the async producer's queue.
> So, ideally if the queue is full then the enqueue operation should result in an kafka.common.QueueFullException in 20 ms.
> The logs indicate that the enqueue operation is taking more than 20ms (takes around 250ms) without throwing any exceptions.
> Is there any other property that could conflict with queue.enqueue.timeout.ms which is causing this behavior ?
> Or is it possible that the queue is not full & yet the producer.send() call is still taking around 200ms under peak load ?
> Also, could you suggest any other alternatives so that we can either enforce a timeout or throw an exception in-case the async producer is taking more than a specified amount of time.
> Regards,
> Dev



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)