[OT] Real Time Open source solutions for aggregation and stream processing

[Ian Holsman <li...@holsman.net>]

firstly, my apologies for the off-topic message,
but I thought most people on this list would be knowledgeable and 
interested in this kind of thing.

We are looking to find a open source, scalable solution to do RT 
aggregation and stream processing (similar to what the 'hop' project 
http://code.google.com/p/hop/ set out to do) for large(ish) click-stream 
logs.

My first thought was something like esper, but in our testing it kind of 
hits the wall at around 10,000 rules per JVM.

I was wondering if any of you guys had some experiences in this area, 
and what your favorite toolsets are around this.

currently we are using cassandra and redis with home grown software to 
do the aggregation, but I'd love to use a common package if there is one.

and again.. apologies for the off-topic message and the x-posting.

regards
Ian