You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@storm.apache.org by "Jungtaek Lim (JIRA)" <ji...@apache.org> on 2017/02/08 02:15:42 UTC
[jira] [Resolved] (STORM-2281) Running Multiple Kafka Spouts
(Trident) Throws Illegal State Exception
[ https://issues.apache.org/jira/browse/STORM-2281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jungtaek Lim resolved STORM-2281.
---------------------------------
Resolution: Fixed
Fix Version/s: 1.1.0
2.0.0
Thanks [~hmclouro], I merged into master and 1.x branch.
> Running Multiple Kafka Spouts (Trident) Throws Illegal State Exception
> ----------------------------------------------------------------------
>
> Key: STORM-2281
> URL: https://issues.apache.org/jira/browse/STORM-2281
> Project: Apache Storm
> Issue Type: Bug
> Components: storm-kafka-client, trident
> Affects Versions: 1.0.0
> Environment: centos mac
> Reporter: Narendra Bidari
> Assignee: Hugo Louro
> Priority: Critical
> Labels: kafka, trident
> Fix For: 2.0.0, 1.1.0
>
> Attachments: Screen Shot 2017-01-09 at 8.54.17 PM.png, Screen Shot 2017-01-30 at 1.44.59 AM.png
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> For Kafka Spout New Consumer in Trident, if we increase the spout parallelism more than one then we can see that the below error happens
> It is reproducible most of the times, it it does not then just kill and restart topology. (if spout parallelism is 1 there is no problem, it only happens with multiple spouts)
> Steps to Reproduce:
> 1. Create a Spout Only Trident Topology (or read write topology)
> 2. Create a topic with multiple partition (2 or more)
> 3. Pump some data and try to read with parallelism of 2 or more
> No current assignment for partition input-1
> at org.apache.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:464) ~[storm-core-1.0.2.jar:1.0.2]
> at org.apache.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:430) ~[storm-core-1.0.2.jar:1.0.2]
> at org.apache.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:73) ~[storm-core-1.0.2.jar:1.0.2]
> at org.apache.storm.daemon.executor$fn__8058$fn__8071$fn__8124.invoke(executor.clj:850) ~[storm-core-1.0.2.jar:1.0.2]
> at org.apache.storm.util$async_loop$fn__624.invoke(util.clj:484) [storm-core-1.0.2.jar:1.0.2]
> at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_77]
> Caused by: java.lang.IllegalStateException: No current assignment for partition input-1
> at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:231) ~[kafka-clients-0.10.0.0.jar:?]
> at org.apache.kafka.clients.consumer.internals.SubscriptionState.seek(SubscriptionState.java:256) ~[kafka-clients-0.10.0.0.jar:?]
> at org.apache.kafka.clients.consumer.KafkaConsumer.seek(KafkaConsumer.java:1134) ~[kafka-clients-0.10.0.0.jar:?]
> at org.apache.storm.kafka.spout.trident.KafkaTridentSpoutEmitter.seek(KafkaTridentSpoutEmitter.java:139) ~[storm-kafka-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
> at org.apache.storm.kafka.spout.trident.KafkaTridentSpoutEmitter.emitPartitionBatch(KafkaTridentSpoutEmitter.java:88) ~[storm-kafka-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
> at org.apache.storm.kafka.spout.trident.KafkaTridentSpoutEmitter.emitPartitionBatch(KafkaTridentSpoutEmitter.java:47) ~[storm-kafka-client-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
> at org.apache.storm.trident.spout.OpaquePartitionedTridentSpoutExecutor$Emitter.emitBatch(OpaquePartitionedTridentSpoutExecutor.java:128) ~[storm-core-1.0.2.jar:1.0.2]
> at org.apache.storm.trident.spout.TridentSpoutExecutor.execute(TridentSpoutExecutor.java:82) ~[storm-core-1.0.2.jar:1.0.2]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)