You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Ben Thompson <t....@gmail.com> on 2015/10/21 22:27:37 UTC

Poor use cases for Spark

Hello,

I'm interested in hearing use cases and parallelism problems where Spark
was *not* a good fit for you. This is an effort to understand the limits of
MapReduce style parallelism.

Some broad things that pop out:
-- recursion
-- problems where the task graph is not known ahead of time
-- some graph problems (
http://www.frankmcsherry.org/graph/scalability/cost/2015/01/15/COST.html)

This is not in anyway an attack on Spark. It's an amazing tool that does
it's job very well. I'm just curious where it starts breaking down. Let me
know if you have any experiences!

Thanks very much,
Ben