You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/07/25 22:12:00 UTC
[jira] [Commented] (SAMZA-1370) Memory leak in CachedStore when
using ByteBufferSerde as key serde
[ https://issues.apache.org/jira/browse/SAMZA-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100859#comment-16100859 ]
ASF GitHub Bot commented on SAMZA-1370:
---------------------------------------
GitHub user prateekm opened a pull request:
https://github.com/apache/samza/pull/251
SAMZA-1370: Memory leak in CachedStore when using ByteBufferSerde as key Serde
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/prateekm/samza bytebufferserde
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/251.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #251
----
commit 5f3a78fb10d93ebdbabb609a9ebeb5df737ffbfa
Author: Prateek Maheshwari <pm...@linkedin.com>
Date: 2017-07-25T22:08:56Z
SAMZA-1370: Memory leak in CachedStore when using ByteBufferSerde as key serde
----
> Memory leak in CachedStore when using ByteBufferSerde as key serde
> ------------------------------------------------------------------
>
> Key: SAMZA-1370
> URL: https://issues.apache.org/jira/browse/SAMZA-1370
> Project: Samza
> Issue Type: Bug
> Reporter: Prateek Maheshwari
> Assignee: Prateek Maheshwari
> Fix For: 0.13.1
>
>
> ByteBufferSerde uses relative bulk get to serialize the provided ByteBuffer which changes its internal position. ByteBuffer's `equals` and `hashCode` depend upon its remaining elements, i.e. on its position. This means that when using ByteBuffers as keys in the CachedStore, flushing cache contents to the underlying store changes their hashCode. Since the hashCode for the key no longer matches the one used when inserting it into the map, the LinkedHashMap cannot correctly evict or remove these entries, leading to a memory leak.
> Changing ByteBufferSerde to use absolute get operations or duplicate the provided ByteBuffer before copying should fix this issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)