Re[2]: Storm Performance

[ch...@gmail.com]

 About the performance of Storm, I have done a performance test. With write/read hbase and redis, my topology could still process about 3000 tuples per second in our StormCluster server.

Sent from  myMail for iOS

2014年1月10日 星期五 23:54 +0800 from Michael Ritsema  <mi...@gmail.com>:
I deploy with LocalCluster mode. I get several thousands messages processed a second but expect my botteneck is not at the storm layer.

It would be interesting to see a full explanation of the differences in LocalCluster and a real deployment. I expect quite a few people would benefit from having storm run in process in actual deployments. 

-Rits

On Fri, Jan 10, 2014 at 9:46 AM, Michael Rose  < michael@fullcontact.com > wrote:
>Post your code.  Even Dev mode is far faster for us.
>On Jan 10, 2014 8:44 AM, "Klausen Schaefersinho" < klaus.schaefers@gmail.com > wrote:
>>>  I've benched storm at 1.8 million tuples per second on a big (24 core) box using local or shuffle grouping between a spout and bolt.  
>>Production or development mode?
>>
>>> you're only seeing 10 events per second make sure you don't have any sleeps Yeah I checked  for sleeps etc. and stripped down my code. Now my bolts do nothing and the spout just creates random data...
>>
>>
>>
>>
>>On Fri, Jan 10, 2014 at 4:37 PM, Nathan Leung  < ncleung@gmail.com > wrote:
>>>I've benched storm at 1.8 million tuples per second on a big (24 core) box using local or shuffle grouping between a spout and bolt. If you're only seeing 10 events per second make sure you don't have any sleeps (whether in your code or elsewhere e.g. a library or triggered due to lack of data in the storm spout). Also note that the default sleep period in the spout when there is no data (in storm 0.9) is I believe 1ms so even if you're hitting this condition you should see much more than 10 events per second.
>>>On Jan 10, 2014 10:19 AM, "Jon Logan" < jmlogan@buffalo.edu > wrote:
>>>>Why would you benchmark local mode? It's intended for debugging and development purposes, not actual production use...............
>>>>
>>>>
>>>>per the website, Storm itself is  benchmarked at > 1 million tuples per second per node.
>>>>
>>>>
>>>>On Fri, Jan 10, 2014 at 8:40 AM, Klausen Schaefersinho  < klaus.schaefers@gmail.com > wrote:
>>>>>Hi,
>>>>>
>>>>>how is the performance of a real cluster compared to a local development cluster? I was trying to benchmark Storm now and found  that it only manages to consume 10 events per second in a simple topology which only consists out of i spout and one bolt where the spout creates random objects and the bolt acknowledge the tuple. 
>>>>>
>>>>>10 events terrible slow, and I am not sure if this is related to the develop mode.
>>>>>
>>>>>Cheers,
>>>>>
>>>>>Klaus
>>