[jira] [Resolved] (HADOOP-19200) Reduce the number of headObject when opening a file with the s3 file system

["Steve Loughran (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-19200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran resolved HADOOP-19200.
-------------------------------------
    Resolution: Duplicate

> Reduce the number of headObject when opening a file with the s3 file system
> ---------------------------------------------------------------------------
>
>                 Key: HADOOP-19200
>                 URL: https://issues.apache.org/jira/browse/HADOOP-19200
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: fs/s3
>    Affects Versions: 3.4.0, 3.3.6
>            Reporter: Oliver Caballero Alvarez
>            Priority: Major
>
> In the implementation of the S3 filesystem, of the hadoop aws package, if you use it with spark, every time you open a file for anything you will have to send two Head Objects, since to open the file, you will first look to see if this file exists, executing a HeadObject, and then when opening it, the implementation, both of sdk1 and sdk2, forces you to make a head object again. This is not the fault of the implementation of this class (S3AFileSystem), but of the abstract FileSystem class of the Hadoop core, since it does not allow the FileStatus to be passed but only allows the use of Path.
> If the FileSystem implementation is changed, it could be used to not have to request that HeadObject again.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org