look for help about Streaming benchmark and how to have a stress testing in Storm

[Andrew Lee <li...@gmail.com>]

Hi, everyone.

I'm a researcher in the distributed system field. Now i'm going to find the
bottlenecks of Storm and Spark Streaming. I'd like to know the popular
benchmarks to test the thoughout and latency of Storm or Spark Streaming.
If someone has experiences or professional knowledge, i'm happy for your
advice.

Another question is how to have a stress testing in Storm. I create a
topology that has a spout (emit sentence) and a bolt that discards the
messages recevied. And the number of spout tasks is 28, of bolts is 28 too.
Of course, the physical resource of servers can satisfy the scenario. In
the running, i observerd the number of emitting sentences is just 500 per
minute per task. I'm so amazed because it can be up to 243,783 messages in
using netty for transferring messages.
I didn't do any advanced configuration. I have no idea how to let the storm
fully using the server resource and then the bottleneck can be found.

Thank you!

Andrew Lee