You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Daryn Sharp (JIRA)" <ji...@apache.org> on 2013/07/31 17:33:50 UTC
[jira] [Commented] (HADOOP-9757) Har metadata cache can grow
without limit
[ https://issues.apache.org/jira/browse/HADOOP-9757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725348#comment-13725348 ]
Daryn Sharp commented on HADOOP-9757:
-------------------------------------
I suppose this approach improves the issue. It still leaks, but just leaks less. I think this can be greatly simplified to just removing the class static cache.
The har AFS delegates to the standard har FS which uses a cached fs. Which means the class static cache is technically unnecessary.
> Har metadata cache can grow without limit
> -----------------------------------------
>
> Key: HADOOP-9757
> URL: https://issues.apache.org/jira/browse/HADOOP-9757
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.0.4-alpha, 0.23.9
> Reporter: Jason Lowe
> Assignee: Cristina L. Abad
> Attachments: HADOOP_9757.branch23.patch
>
>
> MAPREDUCE-2459 added a metadata cache to the har filesystem, but the cache has no upper limits. A long-running process that accesses many har archives will eventually run out of memory due to a har metadata cache that never retires entries.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira