You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Patrick Wendell (JIRA)" <ji...@apache.org> on 2014/10/21 09:47:33 UTC
[jira] [Resolved] (SPARK-735) memory leak in KryoSerializer
[ https://issues.apache.org/jira/browse/SPARK-735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Patrick Wendell resolved SPARK-735.
-----------------------------------
Resolution: Fixed
I think this was fixed a long time ago.
> memory leak in KryoSerializer
> -----------------------------
>
> Key: SPARK-735
> URL: https://issues.apache.org/jira/browse/SPARK-735
> Project: Spark
> Issue Type: Bug
> Affects Versions: 0.7.0
> Reporter: Reynold Xin
> Assignee: Ram Sriharsha
>
> KryoSerializer uses a ThreadLocal object to store a kryo buffer, which is never cleaned up.
> This becomes a serious problem in projects like Shark, where new threads are created constantly by the thrift server. As long as the new thread references the kryo serializer, it will create a new kryo buffer.
> A simple solution is to remove the ThreadLocal reference, and create a new buffer object every time a new kryo serializer instance is created.
> This is not very expensive because large writes actually go through the serialization stream interface, which reuse the buffer anyway.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org