You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by "Ismael Juma (JIRA)" <ji...@apache.org> on 2016/12/21 14:56:58 UTC
[jira] [Commented] (KAFKA-4533) subscribe() then poll() on new
topic is very slow when subscribed to many topics
[ https://issues.apache.org/jira/browse/KAFKA-4533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15767243#comment-15767243 ]
Ismael Juma commented on KAFKA-4533:
------------------------------------
Thanks for the report. The requests are done in batches. We'd need more information to understand the issue. Maybe you can attach more of the log?
> subscribe() then poll() on new topic is very slow when subscribed to many topics
> --------------------------------------------------------------------------------
>
> Key: KAFKA-4533
> URL: https://issues.apache.org/jira/browse/KAFKA-4533
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Affects Versions: 0.10.1.0
> Reporter: Sergey Alaev
>
> Given following case:
> consumer.subscribe(my_new_topic, [249 existing topics])
> publisher.send(my_new_topic, key, value)
> poll(10) until data from my_new_topic arrives
> I see data from `my_new_topic` only after approx. 90 seconds.
> If I subscribe only to my_new_topic, I get results within seconds.
> Logs contain lots of lines like this:
> 19:28:07.972 [kafka-thread] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - Resetting offset for partition demo.com_recipient-2-0 to earliest offset.
> 19:28:08.247 [kafka-thread] DEBUG org.apache.kafka.clients.consumer.internals.Fetcher - Fetched {timestamp=-1, offset=0} for partition demo.com_recipient-2-0
> Probably you should do that in batch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)