[jira] [Commented] (HADOOP-14710) Uber-JIRA: Support AWS Snowball

["Steve Loughran (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-14710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854957#comment-16854957 ] 

Steve Loughran commented on HADOOP-14710:
-----------------------------------------

Full list of REST API issues: https://docs.aws.amazon.com/snowball/latest/ug/using-adapter-supported-api.html

* LIST v2 doesn't work; LIST v1 doesn't do pages, so stops at first 1K under a path. 
* You only get etags on a multipart PUT: better disable change detection. Wonder what happens on a copy.
* does bucket exist v2 API call will probably fail (we haven't adopted that yet though)
* TransferManager is listed as "do not use", but we only use it for uploads and copy; so it's only uploads which are at risk of breaking.

I think at some point soon we need to get rid of xfer manager use, given we know enough about what goes on ourself. Not yet though

> Uber-JIRA: Support AWS Snowball
> -------------------------------
>
>                 Key: HADOOP-14710
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14710
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: John Zhuge
>            Assignee: John Zhuge
>            Priority: Major
>
> Support data transfer between Hadoop and [AWS Snowball|http://docs.aws.amazon.com/snowball/latest/ug/whatissnowball.html].



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org