You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Aaron Fabbri (JIRA)" <ji...@apache.org> on 2016/09/18 22:01:20 UTC
[jira] [Comment Edited] (HADOOP-12876) [Azure Data Lake] Support
for process level FileStatus cache to optimize GetFileStatus frequent
opeations
[ https://issues.apache.org/jira/browse/HADOOP-12876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15501716#comment-15501716 ]
Aaron Fabbri edited comment on HADOOP-12876 at 9/18/16 10:00 PM:
-----------------------------------------------------------------
FYI we're working on on something similar for S3. The umbrella JIRA is HADOOP-13345 and the in-memory implementation is HADOOP-13452. Once this work is complete, it should be easy to refactor so you can access the LocalMetadataStore from ADLS client.
was (Author: fabbri):
FYI we're working on on something similar for S3. The umbrella JIRA is HADOOP-13448 and the in-memory implementation is HADOOP-13452. Once this work is complete, it should be easy to refactor so you can access the LocalMetadataStore from ADLS client.
> [Azure Data Lake] Support for process level FileStatus cache to optimize GetFileStatus frequent opeations
> ---------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-12876
> URL: https://issues.apache.org/jira/browse/HADOOP-12876
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs, fs/azure, tools
> Reporter: Vishwajeet Dusane
> Assignee: Vishwajeet Dusane
>
> Add support to cache GetFileStatus and ListStatus response locally for limited period of time. Local cache for limited period of time would optimize number of calls for GetFileStatus operation.
> One of the example where local limited period cache would be useful - terasort ListStatus on input directory follows with GetFileStatus operation on each file within directory. For 2048 input files in a directory would save 2048 GetFileStatus calls during start up (Using the ListStatus response to cache FileStatus instances).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org