You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Chris Riccomini (JIRA)" <ji...@apache.org> on 2015/03/11 18:36:38 UTC
[jira] [Updated] (SAMZA-591) Slow down reconnects in
KafkaSystemConsumer
[ https://issues.apache.org/jira/browse/SAMZA-591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Riccomini updated SAMZA-591:
----------------------------------
Attachment: SAMZA-591-0.patch
Attaching patch. No RB since it's a one liner. Using the standard 10s pause that we use everywhere else.
> Slow down reconnects in KafkaSystemConsumer
> -------------------------------------------
>
> Key: SAMZA-591
> URL: https://issues.apache.org/jira/browse/SAMZA-591
> Project: Samza
> Issue Type: Bug
> Components: kafka
> Affects Versions: 0.9.0
> Reporter: Chris Riccomini
> Assignee: Chris Riccomini
> Fix For: 0.9.0
>
> Attachments: SAMZA-591-0.patch
>
>
> During a preferred leadership election in Kafka, I see a ton of these messages:
> {noformat}
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> 2015-03-11 17:25:40 GetOffset [INFO] Validating offset 1625488 for topic and partition [topic1,7]
> 2015-03-11 17:25:40 KafkaSystemConsumer [WARN] While refreshing brokers for [topic1,7]: kafka.common.NotLeaderForPartitionException. Retrying.
> {noformat}
> It looks like we are not pausing between retries in KafkaSystemConsumer.refreshBrokers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)