You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Hyukjin Kwon (JIRA)" <ji...@apache.org> on 2019/05/21 04:00:25 UTC
[jira] [Updated] (SPARK-18887) Executor OOM due to tungsten memory
leak in external sorter
[ https://issues.apache.org/jira/browse/SPARK-18887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-18887:
---------------------------------
Labels: bulk-closed (was: )
> Executor OOM due to tungsten memory leak in external sorter
> -----------------------------------------------------------
>
> Key: SPARK-18887
> URL: https://issues.apache.org/jira/browse/SPARK-18887
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.2
> Environment: -master local[*] --driver-memory=14G --conf spark.memory.fraction=0.8, 0.4 and 0.6 show all the same problem
> Reporter: Georg Heiler
> Priority: Major
> Labels: bulk-closed
>
> When performing a collect /write.parquet on http://stackoverflow.com/questions/41169873/spark-dynamic-dag-is-a-lot-slower-and-different-from-hard-coded-dag I can observe problems with Tungstens memory management. The problem is similar to https://issues.apache.org/jira/browse/SPARK-14363 which should be fixed in 2.0.0 - but I am using 2.0.2.
> Playing with spark.memory.fraction did not fix the issue.
> 6/12/15 19:15:56 WARN TaskMemoryManager: leak 64.0 MB memory from org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorter@62d91577
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@264a5b7b in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@18b52a29 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@49965a33 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@44c939b8 in task 15307
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@a074a26 in task 15307
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@7ad8ad95 in task 15312
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@ab0aec8 in task 15307
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@64014c38 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@10e44240 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@21b1d617 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@5a8d57d0 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@e91953 in task 15314
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@4d33cd6d in task 15310
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@30aa29c9 in task 15310
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@69979ca3 in task 15310
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@646416fd in task 15310
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@3403b035 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@64712b7 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@29e97d06 in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@19ea8c3c in task 15308
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@5cc91c89 in task 15312
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@6f792272 in task 15312
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@73a12102 in task 15312
> 16/12/15 19:15:56 WARN TaskMemoryManager: leak a page: org.apache.spark.unsafe.memory.MemoryBlock@37eff7b9 in task 15310
> 16/12/15 19:15:56 ERROR Executor: Exception in task 0.0 in stage 272.0 (TID 15307)
> java.lang.OutOfMemoryError: Unable to acquire 1252 bytes of memory, got 0
> at org.apache.spark.memory.MemoryConsumer.allocatePage(MemoryConsumer.java:129)
> at org.apache.spark.util.collection.unsafe.sort.UnsafeExternalSorter.acquireNewPageIfNecessary(UnsafeExternalSorter.java:377)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org