You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Sung Hwan Chung <co...@cs.stanford.edu> on 2014/06/26 02:13:03 UTC

Number of executors smaller than requested in YARN.

Hi,

When I try requesting a large number of executors - e.g. 242, it doesn't
seem to actually reach that number. E.g., under the executors tab, I only
see an executor ID of upto 234.

This despite the fact that there're plenty more memory available as well as
CPU cores, etc in the system. In fact, in the YARN page, it shows that 243
containers are running (242 executors + driver).

Anyone know what's going on?