You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by gang tian <g1...@gmail.com> on 2026/09/18 07:52:06 UTC

Question about Spark configuration

I am running many Spark 3.3 batch processing jobs on YARN.
How can we configure the appropriate executor memory and off-heap memory
sizes for a Spark 3 offline batch processing job? Setting them too large
wastes resources, while setting them too small may cause insufficient
memory and job failures.