You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (Jira)" <ji...@apache.org> on 2022/01/05 17:05:01 UTC
[jira] [Updated] (HADOOP-16259) Distcp to set S3 Storage Class
[ https://issues.apache.org/jira/browse/HADOOP-16259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Loughran updated HADOOP-16259:
------------------------------------
Parent: HADOOP-18067 (was: HADOOP-17566)
> Distcp to set S3 Storage Class
> ------------------------------
>
> Key: HADOOP-16259
> URL: https://issues.apache.org/jira/browse/HADOOP-16259
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, tools/distcp
> Affects Versions: 2.8.4
> Reporter: Prakash Gopalsamy
> Priority: Minor
> Attachments: ENHANCE_HADOOP_DISTCP_FOR_CUSTOM_S3_STORAGE_CLASS.docx, ENHANCE_HADOOP_DISTCP_FOR_CUSTOM_S3_STORAGE_CLASS.docx.pdf
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Hadoop distcp implementation doesn’t have properties to override Storage class while transferring data to Amazon S3 storage. Hadoop distcp doesn’t set any storage class while transferring data to Amazon S3 storage. Due to this all the objects moved from cluster to S3 using Hadoop Distcp are been stored in the default storage class “STANDARD”. By providing a new feature to override the default S3 storage class through configuration properties will be helpful to upload objects in other storage classes. I have come up with a design to implement this feature in a design document and uploaded the same in the JIRA. Kindly review and let me know for your suggestions.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org