You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@storm.apache.org by James Tyack <jt...@zignallabs.com> on 2016/09/12 18:29:00 UTC

storm-kafka-client 1.0.2 problems: tuples not always being replayed

We're in the process of upgrading from Storm 0.9.5 to 1.0.2.  We're also
upgrading to Kafka 0.10.0.1.

We've been having issues with topologies that fail tuples. The Kafka
offsets seemed to get stuck and the Kafka spout eventually halted even
though the last committed offset was no where near the end of the topic.

My co-worker Jeff isolated the problem to toe storm-kafka-client and has
covered the scenarios with unit tests (and a proposed fix). Details in his
pull request.  Would be grateful if we could hear whether others are having
similar issues / have input on our proposed fix.

https://github.com/apache/storm/pull/1679

Thanks
James