You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Colin Patrick McCabe (JIRA)" <ji...@apache.org> on 2014/02/04 03:19:08 UTC
[jira] [Updated] (HDFS-5810) Unify mmap cache and short-circuit
file descriptor cache
[ https://issues.apache.org/jira/browse/HDFS-5810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Colin Patrick McCabe updated HDFS-5810:
---------------------------------------
Attachment: HDFS-5810.015.patch
Some fixes:
* Don't move tcpReadsDisabledForTesting to BlockReaderFactory, to avoid code churn. For now, leave it in DFSInputStream.
* Add TestBlockReaderFactory to test a few corner cases surrounding the BlockReaderFactory.
* Add new configuration keys to hdfs-default.xml.
* Straighten out a few issues surrounding purging replicas from the cache. Formerly, there were some bugs that could happen if a replica was purged while others held on to references to it.
* Rework the way eviction is handled somewhat. When we run out of mmap slots, we now bump the replicas down into the standard eviction map, rather than purging them immediately. This also motivated me to use eviction maps rather than eviction lists, since otherwise this bumping would be an O(n^2) operation.
* Use a HashMap rather than TreeMap to store replicas.
* Rework ShortCircuitCache#getOrCreateClientMmap and ShortCircuitCache#fetchOrCreate
> Unify mmap cache and short-circuit file descriptor cache
> --------------------------------------------------------
>
> Key: HDFS-5810
> URL: https://issues.apache.org/jira/browse/HDFS-5810
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Affects Versions: 2.3.0
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-5810.001.patch, HDFS-5810.004.patch, HDFS-5810.006.patch, HDFS-5810.008.patch, HDFS-5810.015.patch
>
>
> We should unify the client mmap cache and the client file descriptor cache. Since mmaps are granted corresponding to file descriptors in the cache (currently FileInputStreamCache), they have to be tracked together to do "smarter" things like HDFS-5182.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)