Benchmarking streaming frameworks

[Giselle van Dongen <Gi...@UGent.be>]

Dear users of Streaming Technologies,

As a PhD student in big data analytics, I am currently in the process of
compiling a list of benchmarks (to test multiple streaming frameworks) in
order to create an expanded benchmarking suite. The benchmark suite is being
developed as a part of my current work at Ghent University.

The included frameworks at this time are, in no particular order, Spark,
Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
previous work or relevant benchmarks would be appreciated.

Best regards,
Giselle van Dongen

Re: Benchmarking streaming frameworks

[Dominik Safaric <do...@gmail.com>]

Dear Giselle,

Various stream processing engines benchmarks already exist. Here are only a few of them I believe are worthwhile mentioning:

http://ieeexplore.ieee.org/document/7530084/ <http://ieeexplore.ieee.org/document/7530084/>
https://www.usenix.org/node/188989 <https://www.usenix.org/node/188989>
https://pdfs.semanticscholar.org/c82f/170fbc837291d94dc0a18f0223d182144339.pdf <https://pdfs.semanticscholar.org/c82f/170fbc837291d94dc0a18f0223d182144339.pdf>
https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F14/projects/reports/project11_report_ver3.pdf <https://people.eecs.berkeley.edu/~kubitron/courses/cs262a-F14/projects/reports/project11_report_ver3.pdf>
https://hal.inria.fr/hal-01347638/document <https://hal.inria.fr/hal-01347638/document>

Regards,
Dominik

> On 23 Mar 2017, at 11:09, Giselle van Dongen <Gi...@UGent.be> wrote:
> 
> Dear users of Streaming Technologies,
> 
> As a PhD student in big data analytics, I am currently in the process of
> compiling a list of benchmarks (to test multiple streaming frameworks) in
> order to create an expanded benchmarking suite. The benchmark suite is being
> developed as a part of my current work at Ghent University.
> 
> The included frameworks at this time are, in no particular order, Spark,
> Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
> previous work or relevant benchmarks would be appreciated.
> 
> Best regards,
> Giselle van Dongen

Re: Benchmarking streaming frameworks

[Michael Noll <mi...@confluent.io>]

A recent one is "Analytics on Fast Data: Main-Memory Database Systems
versus Modern Streaming Systems" (
http://db.in.tum.de/~kipf/papers/fastdata.pdf)

For the record, the paper above doesn't yet cover/realize that, nowadays,
the Kafka project includes native stream processing capabilities aka the
Kafka Streams API.

-Michael


On Thu, Mar 23, 2017 at 2:00 PM, Felix Neutatz <ne...@googlemail.com>
wrote:

> Hi,
>
> our team already created a benchmark framework for batch processing
> (including MR,Yarn, Spark, Flink), maybe you like to extend it for
> streaming: https://github.com/peelframework/peel
>
> Best regards,
> Felix
>
>
> On Mar 23, 2017 11:51, "Christophe Salperwyck" <
> christophe.salperwyck@gmail.com> wrote:
>
> Good idea! You could test Akka streams too.
>
> Lots of documents exist:
> https://yahooeng.tumblr.com/post/135321837876/benchmarking-s
> treaming-computation-engines-at
> https://github.com/yahoo/streaming-benchmarks
>
> Cheers,
> Christophe
>
> 2017-03-23 11:09 GMT+01:00 Giselle van Dongen <Gi...@ugent.be>
> :
>
>> Dear users of Streaming Technologies,
>>
>> As a PhD student in big data analytics, I am currently in the process of
>> compiling a list of benchmarks (to test multiple streaming frameworks) in
>> order to create an expanded benchmarking suite. The benchmark suite is
>> being
>> developed as a part of my current work at Ghent University.
>>
>> The included frameworks at this time are, in no particular order, Spark,
>> Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
>> previous work or relevant benchmarks would be appreciated.
>>
>> Best regards,
>> Giselle van Dongen
>>
>
>
>

Re: Benchmarking streaming frameworks

[Felix Neutatz <ne...@googlemail.com>]

Hi,

our team already created a benchmark framework for batch processing
(including MR,Yarn, Spark, Flink), maybe you like to extend it for
streaming: https://github.com/peelframework/peel

Best regards,
Felix

On Mar 23, 2017 11:51, "Christophe Salperwyck" <christophe.salperwyck@gmail.
com> wrote:

Good idea! You could test Akka streams too.

Lots of documents exist:
https://yahooeng.tumblr.com/post/135321837876/benchmarking-
streaming-computation-engines-at
https://github.com/yahoo/streaming-benchmarks

Cheers,
Christophe

2017-03-23 11:09 GMT+01:00 Giselle van Dongen <Gi...@ugent.be>:

> Dear users of Streaming Technologies,
>
> As a PhD student in big data analytics, I am currently in the process of
> compiling a list of benchmarks (to test multiple streaming frameworks) in
> order to create an expanded benchmarking suite. The benchmark suite is
> being
> developed as a part of my current work at Ghent University.
>
> The included frameworks at this time are, in no particular order, Spark,
> Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
> previous work or relevant benchmarks would be appreciated.
>
> Best regards,
> Giselle van Dongen
>

Re: Benchmarking streaming frameworks

[Christophe Salperwyck <ch...@gmail.com>]

Good idea! You could test Akka streams too.

Lots of documents exist:
https://yahooeng.tumblr.com/post/135321837876/benchmarking-streaming-computation-engines-at
https://github.com/yahoo/streaming-benchmarks

Cheers,
Christophe

2017-03-23 11:09 GMT+01:00 Giselle van Dongen <Gi...@ugent.be>:

> Dear users of Streaming Technologies,
>
> As a PhD student in big data analytics, I am currently in the process of
> compiling a list of benchmarks (to test multiple streaming frameworks) in
> order to create an expanded benchmarking suite. The benchmark suite is
> being
> developed as a part of my current work at Ghent University.
>
> The included frameworks at this time are, in no particular order, Spark,
> Flink, Kafka (Streams), Storm (Trident) and Drizzle. Any pointers to
> previous work or relevant benchmarks would be appreciated.
>
> Best regards,
> Giselle van Dongen
>