[jira] [Commented] (KAFKA-1111) Broker prematurely accepts TopicMetadataRequests on startup

["Jason Rosenberg (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13815088#comment-13815088 ] 

Jason Rosenberg commented on KAFKA-1111:
----------------------------------------

I've now removed the broker from the metatdata vip rotatation, for a delay period (e.g. 20 seconds), and this does seem to help reduce the preponderance of these Topic creation log messages.

However, there are still some of these coming through occasionally.  It seems this happens when there is a fetch request coming from a consumer.  Which I can't block (since consumers don't use the metadata request path, they just use zookeeper).  So, I'm guessing this would also be fixed by not allowing the server to start until it's made an attempt to sync up it's metadata.

Here's an email from the users forum:

me: Could a fetch request from a consumer cause a Topic creation request
Neha: Yes, that seems like a way the broker can get into this situation.


> Broker prematurely accepts TopicMetadataRequests on startup
> -----------------------------------------------------------
>
>                 Key: KAFKA-1111
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1111
>             Project: Kafka
>          Issue Type: Bug
>            Reporter: Jason Rosenberg
>            Assignee: Neha Narkhede
>
> I have an issue where on startup, the broker starts accepting TopicMetadataRequests before it has had metadata sync'd from the controller.  This results in a bunch of log entries that look like this:
> 013-11-01 03:26:01,577  INFO [kafka-request-handler-0] admin.AdminUtils$ - Topic creation { "partitions":{ "0":[ 9, 10 ] }, "version":1 }
> 2013-11-01 03:26:07,767  INFO [kafka-request-handler-1] admin.AdminUtils$ - Topic creation { "partitions":{ "0":[ 9, 11 ] }, "version":1 }
> 2013-11-01 03:26:07,823  INFO [kafka-request-handler-1] admin.AdminUtils$ - Topic creation { "partitions":{ "0":[ 10, 11 ] }, "version":1 }
> 2013-11-01 03:26:11,183  INFO [kafka-request-handler-2] admin.AdminUtils$ - Topic creation { "partitions":{ "0":[ 10, 11 ] }, "version":1 }
> From an email thread, Neha remarks:
> Before a broker receives the first
> LeaderAndIsrRequest/UpdateMetadataRequest, it is technically not ready to
> start serving any request. But it still ends up serving
> TopicMetadataRequest which can re-create topics accidentally. It shouldn't
> succeed, but this is still a problem.



--
This message was sent by Atlassian JIRA
(v6.1#6144)