Most bolts are idle, and the upstream bolt stopped pulling data

[Chen Wang <ch...@gmail.com>]

Hi,
i am having a bolt to poll from kafka(bolt, not spout). It has 3 threads as
the topic has 3 partitions. Then the downstream processing bolt has 200
threads. With this set up I hope the processing bolt can catch up with the
kafka bolt. However, in my testing, it seems that at a single time, only 3
processing bolts are processing the data, and then the kafka polling bolt
seems to be blocked.
I am simply using:

BatchProcessorBolt batchProcessorBolt = new BatchProcessorBolt();

 builder.setBolt("processing bolt),

  batchProcessorBolt, 200)

  .shuffleGrouping("kafkapollingbolt");


and kafkapollingbolt simply polls from kafka topic.

pollingBolt = new KafkaPollingBolt(kafkaConsumerProperties,

  kafkaTopic);


Is there something apparently missing here? If I only have
kafkaPollingBolt, then the speed is expected. But once adding the
processorBolt, kafkaPollingBolt will also hangs.

Thanks in advance.

Chen