You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "Wenchen Fan (JIRA)" <ji...@apache.org> on 2018/02/02 04:23:00 UTC

[jira] [Resolved] (SPARK-23306) Race condition in TaskMemoryManager

     [ https://issues.apache.org/jira/browse/SPARK-23306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Wenchen Fan resolved SPARK-23306.
---------------------------------
       Resolution: Fixed
    Fix Version/s: 2.4.0

Issue resolved by pull request 20480
[https://github.com/apache/spark/pull/20480]

> Race condition in TaskMemoryManager
> -----------------------------------
>
>                 Key: SPARK-23306
>                 URL: https://issues.apache.org/jira/browse/SPARK-23306
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.2.1
>            Reporter: Zhan Zhang
>            Priority: Minor
>             Fix For: 2.4.0
>
>
> There is race condition in TaskMemoryManger, which may cause OOM.
> The memory released may be taken by another task because there is a gap between releaseMemory and acquireMemory, e.g., UnifiedMemoryManager, causing the OOM. if the current is the only one that can perform spill. It can happen to BytesToBytesMap, as it only spill required bytes.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org