[jira] [Resolved] (KAFKA-665) Outgoing responses delayed on a busy Kafka broker

["Jay Kreps (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KAFKA-665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jay Kreps resolved KAFKA-665.
-----------------------------
    Resolution: Fixed

We solved this by making the response queue unbounded. When the request queue is full the network thread will block until some space frees up, but there is no deadlock here, processors can still make progress. We do need to block the processors to avoid continuing to read and enqueue an unbounded number of new requests. You could imagine being smarter about this but I don't think the current situation is too bad.

> Outgoing responses delayed on a busy Kafka broker 
> --------------------------------------------------
>
>                 Key: KAFKA-665
>                 URL: https://issues.apache.org/jira/browse/KAFKA-665
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8.0
>            Reporter: Neha Narkhede
>            Priority: Critical
>              Labels: replication-performance
>             Fix For: 0.9.0
>
>
> In a long running test, I observed that after a few hours of operation, few requests start timing out, mainly because they spent very long time sitting in the response queue -
> [2012-12-07 22:05:56,670] TRACE Completed request with correlation id 3965966 and client : TopicMetadataRequest:4009, queueTime:1, localTime:28, remoteTime:0, sendTime:3980 (kafka.network.RequestChannel$)
> [2012-12-07 22:04:12,046] TRACE Completed request with correlation id 3962561 and client : TopicMetadataRequest:3449, queueTime:0, localTime:29, remoteTime:0, sendTime:3420 (kafka.network.RequestChannel$)
> [2012-12-07 22:05:56,670] TRACE Completed request with correlation id 3965966 and client : TopicMetadataRequest:4009, queueTime:1, localTime:28, remoteTime:0, sendTime:3980 (kafka.network.RequestChannel$)
> We might have a problem in the way we process outgoing responses. Basically, if the processor thread blocks on enqueuing requests in the request queue, it doesn't come around to processing its responses which are ready to go out. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)