You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2014/05/21 08:55:38 UTC
[jira] [Commented] (MAPREDUCE-5896) Allow InputSplits to indicate
which locations have the block cached in memory
[ https://issues.apache.org/jira/browse/MAPREDUCE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004424#comment-14004424 ]
Sandy Ryza commented on MAPREDUCE-5896:
---------------------------------------
Given HDFS's plans for hierarchical storage management, I think it would be good to make this extensible to handle storage mediums beyond memory. I talked this over with [~andrew.wang] and [~atm] and we think the right interface would be something like a SplitLocationInfo class, with isInMemory() and isOnDisk() methods. We can later add isInFlash() and possibly even getDisk() to return which disk the data is on.
InputSplits would have a "SplitLocationInfo[] getLocationInfo()" method that returns info about how the data is stored on each host returned by getLocations().
> Allow InputSplits to indicate which locations have the block cached in memory
> -----------------------------------------------------------------------------
>
> Key: MAPREDUCE-5896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5896
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 2.4.0
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
>
--
This message was sent by Atlassian JIRA
(v6.2#6252)