You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Jay Kreps (JIRA)" <ji...@apache.org> on 2011/08/15 00:02:27 UTC

[jira] [Commented] (KAFKA-48) Make fetcher blocking

    [ https://issues.apache.org/jira/browse/KAFKA-48?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084912#comment-13084912 ] 

Jay Kreps commented on KAFKA-48:
--------------------------------

This latency issue is important for replication because the latency for the producer will now depend on the replication (fetching) on the followers. This means our current polling mechanism is not going to be good, because we have to either back off for a period of time to avoid busy waiting on the server. In addition with replication we need a similar ability to process requests asynchronously--we do not want to block any threads while waiting for acks from followers. This also breaks our simple request/response model.

This is also important for the streaming use cases as they involve a large number of stacked topics, and hence the end-to-end latency is a multiple of the single-hop consumer latency.

Fixing this is slightly tricky.

The first thing I think we would need to do is move the execution of request handling out of the socket server threads. This would generally be a good thing to do anyway, as I/O currently blocks request handling for all sockets sharing a thread. This can add unnecessary latency.

The design for this could be a request BlockingQueue that the SocketServer submits all requests to, and N response BlockingQueues, one for each socket thread. The request processing would happen in a separate threadpool that would feed off the request queue and send responses back to the response queue. For asynchronous requests, no response need be enqueued.

The request handling would now be an ExecutorService with a fixed number of processes. Each process would poll the request queue, process any request it gets, and send back responses.

Long poll requests from fetchers would either be handled immediately, or, if there is no available data, would add themselves to a list of watchers on the topic. When a request comes in on that topic, it would and responses for all watchers. The request would specify a max timeout after which the request would return empty, this could be implemented with a DelayQueue that was checked periodically for expired requests. A generalization of this would be to have the fetch request provide not only a max_wait_time but also a min_data_size, which would make the request block until the given number of bytes of data have accumulated. This would actually enable the opposite of simple long poll--instead of trying to minimize latency the fetcher would be able to ensure they got a good size chunk of data on each request to ensure good throughput and avoid lots of little requests each fetching only a few small messages.

A similar mechanism would be possible for acknowledgements coming from followers. When a produce request occurs with a min_ack_count > 1, the request would go into a list of waiting requests for that topic/partition. When the ack request comes in from followers, we would check the waiting producers and add responses to the response queue for any newly unblocked request.

I would like to do a round of refactoring on the SocketServer anyway, so let me know if anyone has comments on this before i go do anything too crazy. For example, I want someone else to validate the interaction with the replication design.


> Make fetcher blocking
> ---------------------
>
>                 Key: KAFKA-48
>                 URL: https://issues.apache.org/jira/browse/KAFKA-48
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Alan Cabrera
>
> Currently, the fetch request is non-blocking. If there is nothing on the broker for the consumer to retrieve, the broker simply returns an empty set to the consumer. This can be inefficient. We should make a blocking version of the fetcher so that the fetch request is not returned until the broker has at least one message for the fetcher.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira