[jira] [Commented] (KAFKA-4571) Consumer fails to retrieve messages if started before producer

["Ewen Cheslack-Postava (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-4571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15786707#comment-15786707 ] 

Ewen Cheslack-Postava commented on KAFKA-4571:
----------------------------------------------

Since another consumer group ID consumes the message, I'm assuming you're using auto.offset.reset = earliest?

Does the consumer *never* consume messages, or do you just need to wait long enough? If you start the consumer before the topic exists, it will get metadata indicating there are no partitions. I think you'd ned to wait for the metadata timeout before it would try to refresh the metadata and finally see the created topic partitions (default metadata.max.age.ms is 5 minutes).

> Consumer fails to retrieve messages if started before producer
> --------------------------------------------------------------
>
>                 Key: KAFKA-4571
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4571
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients
>    Affects Versions: 0.10.1.1
>         Environment: Ubuntu Desktop 16.04 LTS, Oracle Java 8 1.8.0_101, Core i7 4770K
>            Reporter: Sergiu Hlihor
>
> In a configuration where topic was never created before, starting the consumer before the producer leads to no message being consumed (KafkaConsumer.pool() returns always an instance of ConsumerRecords with 0 count ). 
> Starting another consumer on the same group, same topic after messages were produced is still not consuming them. Starting another consumer with another groupId appears to be working.
> In the consumer logs I see: WARN  NetworkClient - Error while fetching metadata with correlation id 1 : {measurements021=LEADER_NOT_AVAILABLE} 
> Both producer and consumer were launched from inside same JVM. 
> The configuration used is the standard one found in Kafka distribution. If this is a configuration issue, please suggest any change that I should do.
> Thank you



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)