You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2019/10/22 14:33:00 UTC
[jira] [Commented] (HADOOP-16629) support copyFile in s3a
filesystem
[ https://issues.apache.org/jira/browse/HADOOP-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957120#comment-16957120 ]
Steve Loughran commented on HADOOP-16629:
-----------------------------------------
To add a comment here spanning multiple PRs: we are adding a new API call which should be implementable by all object stores. Furthermore, given it is intended to be used by applications external to the hadoop-* JARs, it's going to have to be specified and tested and implicitly comes with some promise that we will not delete it on whim.
We need to get it right -or at least think we have something which meets these requirements, rather than just exposing the AWS S3 API and saying "here".
In particular, I want the API to be restricted to operations within the same file system instance. I know S3 COPY lets you copy across stores, but there is complexity you have not yet discovered-especially in terms of the features of the S3A client and how they relate. That includes encryption, s3guard, delegation tokens and other advanced features.
Please do not try to implement a cross FS API just because the AWS API offers it. You will only complicate the lives of yourself and others-and I fear I am one of the others.
+[~gabor.bota][~gopalv]
> support copyFile in s3a filesystem
> ----------------------------------
>
> Key: HADOOP-16629
> URL: https://issues.apache.org/jira/browse/HADOOP-16629
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.2.1
> Reporter: Rajesh Balamohan
> Assignee: Rajesh Balamohan
> Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org