Re: look for help about Streaming benchmark and how to have a stress testing in Storm

[Andrew Lee <li...@gmail.com>]

That's very helpful! Thank you very much!

Andrew Lee


2015-03-26 22:38 GMT+08:00 Grant Overby (groverby) <gr...@cisco.com>:

>   I found the following link useful in a similar situation. It doesn’t
> sound like you’re using trident, but I expect at least some of the content
> will still be relevant.
>
>  https://gist.github.com/mrflip/5958028
>
> *Grant Overby*
> Software Engineer
> Cisco.com <http://www.cisco.com/>
> groverby@cisco.com
> Mobile: *865 724 4910*
>
>
>
>        Think before you print.
>
> This email may contain confidential and privileged material for the sole
> use of the intended recipient. Any review, use, distribution or disclosure
> by others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by
> reply email and delete all copies of this message.
>
> Please click here
> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for
> Company Registration Information.
>
>
>
>
>   From: Andrew Lee <li...@gmail.com>
> Reply-To: "user@storm.apache.org" <us...@storm.apache.org>
> Date: Thursday, March 26, 2015 at 3:45 AM
> To: "user@storm.apache.org" <us...@storm.apache.org>
> Subject: look for help about Streaming benchmark and how to have a stress
> testing in Storm
>
>   Hi, everyone.
>
>  I'm a researcher in the distributed system field. Now i'm going to find
> the bottlenecks of Storm and Spark Streaming. I'd like to know the popular
> benchmarks to test the thoughout and latency of Storm or Spark Streaming.
> If someone has experiences or professional knowledge, i'm happy for your
> advice.
>
>  Another question is how to have a stress testing in Storm. I create a
> topology that has a spout (emit sentence) and a bolt that discards the
> messages recevied. And the number of spout tasks is 28, of bolts is 28 too.
> Of course, the physical resource of servers can satisfy the scenario. In
> the running, i observerd the number of emitting sentences is just 500 per
> minute per task. I'm so amazed because it can be up to 243,783 messages in
> using netty for transferring messages.
> I didn't do any advanced configuration. I have no idea how to let the
> storm fully using the server resource and then the bottleneck can be found.
>
>  Thank you!
>
>  Andrew Lee
>



-- 
*Andrew Lee*
from ICT of Chinese Academy of Sciences

Life For Joy
微笑着 追逐每一份阳光