Advanced GC Tuning

["Kuznetsov, Oleksandr" <ol...@deloitte.com.INVALID>]

Hello,

I was reading the Garbage Collection Tuning guide here: Tuning - Spark 3.1.2 Documentation (apache.org)<https://spark.apache.org/docs/3.1.2/tuning.html#garbage-collection-tuning>, specifically section on "Advanced GC Tuning". It is stated that if OldGen region is getting full, it is recommended to lower spark.memory.fraction. I am wondering if this would lower the overall amount of memory available for both storage and execution, slowing down execution. Isn't it better to lower spark.memory.storageFraction instead?  In this case there is less memory available for caching objects, while execution is not being affected. Please see below the copy of the passage I am referring to:


*       "In the GC stats that are printed, if the OldGen is close to being full, reduce the amount of memory used for caching by lowering spark.memory.fraction; it is better to cache fewer objects than to slow down task execution. Alternatively, consider decreasing the size of the Young generation. This means lowering -Xmn if you've set it as above. If not, try changing the value of the JVM's NewRatio parameter. Many JVMs default this to 2, meaning that the Old generation occupies 2/3 of the heap. It should be large enough such that this fraction exceeds spark.memory.fraction."

I would greatly appreciate if you could clarify it for me.

Thank you in advance.

Best regards,
Oleksandr (Alex) Kuznetsov
DC Solution Specialist | Consulting/Strategy and Analytics
Deloitte Consulting LLP
1001 Heathrow Park Ln, Lake Mary, FL 32746
Mobile: 281-384-1331
olkuznetsov@deloitte.com<ma...@deloitte.com> | www.deloitte.com<http://www.deloitte.com/>


This message (including any attachments) contains confidential information intended for a specific individual and purpose, and is protected by law. If you are not the intended recipient, you should delete this message and any disclosure, copying, or distribution of this message, or the taking of any action based on it, by you is strictly prohibited.

Deloitte refers to a Deloitte member firm, one of its related entities, or Deloitte Touche Tohmatsu Limited ("DTTL"). Each Deloitte member firm is a separate legal entity and a member of DTTL. DTTL does not provide services to clients. Please see www.deloitte.com/about to learn more.

v.E.1

Re: Advanced GC Tuning

[Sean Owen <sr...@gmail.com>]

You're right, I think storageFraction is somewhat better to control this,
although some things 'counted' in spark.memory.fraction will also be
long-lived and in the OldGen.
You can also increase the OldGen size if you're pretty sure that's the
issue - 'old' objects in the YoungGen.

I'm not sure how much these will affect performance with modern JVMs; this
advice is 5-9 years old.

On Tue, Jul 20, 2021 at 5:39 PM Kuznetsov, Oleksandr
<ol...@deloitte.com.invalid> wrote:

> Hello,
>
>
>
> I was reading the Garbage Collection Tuning guide here: Tuning - Spark
> 3.1.2 Documentation (apache.org)
> <https://spark.apache.org/docs/3.1.2/tuning.html#garbage-collection-tuning>,
> specifically section on “Advanced GC Tuning”. It is stated that if OldGen
> region is getting full, it is recommended to lower *spark.memory.fraction*.
> I am wondering if this would lower the overall amount of memory available
> for both storage and execution, slowing down execution. Isn’t it better to
> lower *spark.memory.storageFraction* instead?  In this case there is less
> memory available for caching objects, while execution is not being
> affected. Please see below the copy of the passage I am referring to:
>
>
>
> ·       “In the GC stats that are printed, if the OldGen is close to
> being full, reduce the amount of memory used for caching by lowering
> spark.memory.fraction; it is better to cache fewer objects than to slow
> down task execution. Alternatively, consider decreasing the size of the
> Young generation. This means lowering -Xmn if you’ve set it as above. If
> not, try changing the value of the JVM’s NewRatio parameter. Many JVMs
> default this to 2, meaning that the Old generation occupies 2/3 of the
> heap. It should be large enough such that this fraction exceeds
> spark.memory.fraction.”
>
> I would greatly appreciate if you could clarify it for me.
>
>
>