[jira] [Commented] (HADOOP-10587) Use a thread-local cache in TokenIdentifier#getBytes to avoid creating many DataOutputBuffer objects

["Colin Patrick McCabe (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-10587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13993079#comment-13993079 ] 

Colin Patrick McCabe commented on HADOOP-10587:
-----------------------------------------------

Recently, I encountered an edit log with many {{DELEGATION_TOKEN_IDENTIFIER}} log entries.  Attempts to load this edit log failed with an out-of-memory error, occurring in {{TokenIdentifier#getBytes}}.  While the edit log seemed to have an unusally large number of these log entries due to a misconfiguration, I think it would also be useful to optimize {{TokenIdentifier#getBytes}} to reduce this memory consumption somewhat.

> Use a thread-local cache in TokenIdentifier#getBytes to avoid creating many DataOutputBuffer objects
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-10587
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10587
>             Project: Hadoop Common
>          Issue Type: Improvement
>    Affects Versions: 2.4.0
>            Reporter: Colin Patrick McCabe
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>         Attachments: HADOOP-10587.001.patch
>
>
> We can use a thread-local cache in TokenIdentifier#getBytes to avoid creating many DataOutputBuffer objects.  This will reduce our memory usage (for example, when loading edit logs), and help prevent OOMs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)