Empty topic metadata returned during/shortly after server startup

[David Corley <da...@gmail.com>]

Hey all,
We're trying to write some integration tests around a Ruby-based Kafka
client we're developing that leverages both poseidon and poseidon_cluster
gems. We're running Kafka 0.8.0 in a single node config with a single ZK
instance supporting it on the same machine.

The basic tests is as follows:
1. Start a single ZK node and single Kafka broker
2. Produce a set of 20 messages to the broker
3. Initiate a message fetch loop from our client
4. Restart the Kafka node (we're testing error handling)

Step 3 is really a two part request if you look inside the poseidon gem.
The first part is for the topic metadata of the given topic. The second
part is then to fetch the messages from the topic. When we run step 4 we
find that we first hit a connection exception (which we handle in the form
of a retry). When the retry occurs, it appears to be succesful, but closer
inspection reveals and issue with the topic metadata.

When de-bugging, we've found the the first request occasionally returns a
empty topic metadata response with no error code. Looking at the poseidon
source, we can't see any way this could happen unless Kafka itself is
sending back empty topic metadata.

 There does appear to be a significant delay between the time port 9092 is
ready to receive connections and what we consider to be the log message
that signifies the broker is ready:
[2015-04-07 12:42:05,732] INFO Awaiting socket connections on 0.0.0.0:9092.
(kafka.network.Acceptor)
...
[2015-04-07 12:42:18,079] INFO [Kafka Server 1], Started
(kafka.server.KafkaServer)


Is there any circumstances under which Kafka could send back empty topic
metadata for a topic that exists, particularly during broker startup?