You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/12/09 16:23:00 UTC

[jira] [Commented] (HADOOP-14661) S3A to support Requester Pays Buckets

    [ https://issues.apache.org/jira/browse/HADOOP-14661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16714011#comment-16714011 ] 

Steve Loughran commented on HADOOP-14661:
-----------------------------------------

Still want to get this in, still hoping someone adds tests. I think some of the common crawl data is requester pays

Here's the help I need
# Someone needs to write a test to try and read requester-pays data, *before this patch goes in*
# Error message you get is noted, added to troubleshooting.md, and we make sure that the S3ARetryPolicy does not attempt to retry here
# then apply this patch, verify test
# then add an extended test: seek around a requester pays file, first to EOF-1, read a byte, then seek to 0 and read another byte. This is needed to verify that GET requests triggered within in an input stream always sets the header

> S3A to support Requester Pays Buckets
> -------------------------------------
>
>                 Key: HADOOP-14661
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14661
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: common, util
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Mandus Momberg
>            Assignee: Mandus Momberg
>            Priority: Minor
>         Attachments: HADOOP-14661.patch
>
>   Original Estimate: 2h
>  Remaining Estimate: 2h
>
> Amazon S3 has the ability to charge the requester for the cost of accessing S3. This is called Requester Pays Buckets. 
> In order to access these buckets, each request needs to be signed with a specific header. 
> http://docs.aws.amazon.com/AmazonS3/latest/dev/RequesterPaysBuckets.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org