You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Guillermo Ortiz <ko...@gmail.com> on 2018/08/30 21:00:48 UTC

Local mode vs client mode with one executor

I have many spark processes, some of them are pretty simple and they don't
have to process almost messages but they were developed with the same
archeotype and they use spark.

Some of them are executed with many executors but a few ones don't make
sense to process with more than 2-4 cores in only one executor. The most
important reason is that the quantity of messages is so low,, that it's not
worth it.

The point here it's,, any disventage if I run this few spark processes in
local[2..4] instance of cluster/client mode with one executor (4 cores) and
one driver?. I have read that it's a testing mode in most of cases and I
use for my tests too ;)
Besides, it seems that it goes faster running in local mode in those cases.