You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Marcelo Vanzin (JIRA)" <ji...@apache.org> on 2019/05/21 17:24:00 UTC
[jira] [Resolved] (SPARK-27726) Performance of InMemoryStore
suffers under load
[ https://issues.apache.org/jira/browse/SPARK-27726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Vanzin resolved SPARK-27726.
------------------------------------
Resolution: Fixed
Fix Version/s: 3.0.0
Issue resolved by pull request 24616
[https://github.com/apache/spark/pull/24616]
> Performance of InMemoryStore suffers under load
> -----------------------------------------------
>
> Key: SPARK-27726
> URL: https://issues.apache.org/jira/browse/SPARK-27726
> Project: Spark
> Issue Type: Umbrella
> Components: Spark Core
> Affects Versions: 2.3.3
> Reporter: David C Navas
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: GCRateIssues.pdf, PerformanceBeforeAndAfter.pdf
>
>
> When our Spark system has been under load for an extended period of time, GC remains highly active and the jobs page becomes unresponsive even when load eases. Please see the attached GCRateIssues for more details regarding the problem definition.
> We found a number of separate issues which are detailed in the subtasks. I anticipate committing a single PR for all subtasks whose commits roughly align with the descriptions in the subtasks.
> The performance of the code is measured before and after the change and is attached in the document PerformanceBeforeAndAfter. tl;dr in our use case, we saw about five (computed) orders of magnitude improvement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org