You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Gian Merlino (JIRA)" <ji...@apache.org> on 2015/03/18 06:46:39 UTC
[jira] [Updated] (SAMZA-606) KafkaSystemConsumer can stop reading
topic-partitions after broker restarts
[ https://issues.apache.org/jira/browse/SAMZA-606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gian Merlino updated SAMZA-606:
-------------------------------
Affects Version/s: 0.8.0
> KafkaSystemConsumer can stop reading topic-partitions after broker restarts
> ---------------------------------------------------------------------------
>
> Key: SAMZA-606
> URL: https://issues.apache.org/jira/browse/SAMZA-606
> Project: Samza
> Issue Type: Bug
> Affects Versions: 0.8.0
> Reporter: Gian Merlino
>
> Recently I restarted a broker and that caused a couple of partitions to stop being read by a Samza job. I think what happened is that the BrokerProxy found that the broker was no longer leader for the partition, which triggered a refresh, which ended up trying to pull metadata from that same broker. The broker had just started back up and hadn't populated its topic metadata cache yet, so it didn't return anything for that partition and then Samza dropped it.
> There were three log messages for the partition around that time that I think back this story up:
> 2015-03-16 18:04:26,729 INFO org.apache.samza.system.kafka.KafkaSystemConsumer - Abdicating for [xxx,41]
> 2015-03-16 18:04:26,730 INFO kafka.client.ClientUtils$ - Fetching metadata from broker id:1,host:10.186.151.154,port:9092 with correlation id 31 for 1 topic(s) Set(xxx)
> 2015-03-16 18:04:26,731 WARN org.apache.samza.system.kafka.KafkaSystemConsumer - No such topic partition: [xxx,41], dropping.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)