You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@storm.apache.org by "Erik Weathers (JIRA)" <ji...@apache.org> on 2017/12/14 00:56:00 UTC
[jira] [Updated] (STORM-2361) Kafka spout - after topic leader
change, it stops committing offsets to ZK
[ https://issues.apache.org/jira/browse/STORM-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Weathers updated STORM-2361:
---------------------------------
Summary: Kafka spout - after topic leader change, it stops committing offsets to ZK (was: Kafka spout - after leader change, it stops committing offsets to ZK)
> Kafka spout - after topic leader change, it stops committing offsets to ZK
> --------------------------------------------------------------------------
>
> Key: STORM-2361
> URL: https://issues.apache.org/jira/browse/STORM-2361
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka
> Affects Versions: 1.1.0
> Reporter: Ernestas Vaiciukevičius
> Assignee: Ernestas Vaiciukevičius
> Fix For: 2.0.0, 1.1.0, 1.0.4
>
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> After STORM-2296 although Kafka spouts do not generate duplicates, the offsets committment to ZK may stop on recreated PartitionManagers.
> This is because ack's for messages emitted by already destroyed PartitionManagers are not routed properly to the new PartitionManagers handling that partition.
> E.g:
> {code:java} public void ack(Object msgId) {
> KafkaMessageId id = (KafkaMessageId) msgId;
> PartitionManager m = _coordinator.getManager(id.partition);
> if (m != null) {
> m.ack(id.offset);
> }
> {code}
> id.partition is Partition(host, partition, topic), which is different if Kafka broker changed.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)