You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Chris Douglas (JIRA)" <ji...@apache.org> on 2017/11/28 23:55:01 UTC
[jira] [Created] (MAPREDUCE-7016) Avoid making separate RPC calls
for FileStatus and block locations in FileInputFormat
Chris Douglas created MAPREDUCE-7016:
----------------------------------------
Summary: Avoid making separate RPC calls for FileStatus and block locations in FileInputFormat
Key: MAPREDUCE-7016
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7016
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Chris Douglas
{{FileInputFormat::getSplits}} uses {{FileSystem::globStatus}} to determine its inputs. When the glob returns directories, each is traversed and {{LocatedFileStatus}} instances are returned with the block locations. However, when the glob returns files, each requires a second RPC to obtain its locations.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org