You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2014/05/21 08:55:38 UTC

[jira] [Commented] (MAPREDUCE-5896) Allow InputSplits to indicate which locations have the block cached in memory

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14004424#comment-14004424 ] 

Sandy Ryza commented on MAPREDUCE-5896:
---------------------------------------

Given HDFS's plans for hierarchical storage management, I think it would be good to make this extensible to handle storage mediums beyond memory.  I talked this over with [~andrew.wang] and [~atm] and we think the right interface would be something like a SplitLocationInfo class, with isInMemory() and isOnDisk() methods.  We can later add isInFlash() and possibly even getDisk() to return which disk the data is on. 

InputSplits would have a "SplitLocationInfo[] getLocationInfo()" method that returns info about how the data is stored on each host returned by getLocations().

> Allow InputSplits to indicate which locations have the block cached in memory
> -----------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5896
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5896
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 2.4.0
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>




--
This message was sent by Atlassian JIRA
(v6.2#6252)