You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2016/08/19 18:51:20 UTC
[jira] [Commented] (HADOOP-13321) Deprecate FileSystem APIs that
promote inefficient call patterns.
[ https://issues.apache.org/jira/browse/HADOOP-13321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15428635#comment-15428635 ]
Steve Loughran commented on HADOOP-13321:
-----------------------------------------
Having been through our own code, {{exists()}} is used in places where it would be really convoluted to replace it with getFileStatus and exception catching. We only really want to discourage it when used back to back with other get file status calls, or before an operation (delete, rename) which checks some of the same entities.
If the FS client does some very short term caching, even a fraction of a second, the penalty of two back-to-back getFileStatus() checks would become zero ... that may be the way to go.
> Deprecate FileSystem APIs that promote inefficient call patterns.
> -----------------------------------------------------------------
>
> Key: HADOOP-13321
> URL: https://issues.apache.org/jira/browse/HADOOP-13321
> Project: Hadoop Common
> Issue Type: Improvement
> Components: fs
> Reporter: Chris Nauroth
> Assignee: Mingliang Liu
> Attachments: HADOOP-13321.000.patch
>
>
> {{FileSystem}} contains several methods that act as convenience wrappers over calling {{getFileStatus}} and retrieving a single property of the returned {{FileStatus}}. These methods have a habit of fostering inefficient call patterns in applications, resulting in multiple redundant {{getFileStatus}} calls. For HDFS, this translates into wasteful NameNode RPC traffic. For file systems backed by cloud object stores, this translates into wasteful HTTP traffic. This issue proposes to deprecate these methods and instead encourage applications to call {{getFileStatus}} and then reuse the same {{FileStatus}} instance as needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org