You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Daniel Carl Jones (Jira)" <ji...@apache.org> on 2022/04/05 14:08:00 UTC
[jira] [Comment Edited] (HADOOP-14837) Handle S3A "glacier" data
[ https://issues.apache.org/jira/browse/HADOOP-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17516689#comment-17516689 ]
Daniel Carl Jones edited comment on HADOOP-14837 at 4/5/22 2:07 PM:
--------------------------------------------------------------------
FYI, Glacier recently introduced a new 'instant' storage class offering millisecond retrieval: [https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-s3-glacier-instant-retrieval-storage-class/]
We may instead want some way to indicate if a file is available immediately or needs requesting, and this might influence our choice of design.
was (Author: JIRAUSER284792):
FYI, Glacier recently introduced a new 'instant' storage class offering millisecond retrieval: [https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-s3-glacier-instant-retrieval-storage-class/]
We may instead want some way to indicate if a file is available immediately or needs requesting, and this might influence our choice of design.
> Handle S3A "glacier" data
> -------------------------
>
> Key: HADOOP-14837
> URL: https://issues.apache.org/jira/browse/HADOOP-14837
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.0.0-beta1
> Reporter: Steve Loughran
> Priority: Minor
>
> SPARK-21797 covers how if you have AWS S3 set to copy some files to glacier, they appear in the listing but GETs fail, and so does everything else
> We should think about how best to handle this.
> # report better
> # if listings can identify files which are glaciated then maybe we could have an option to filter them out
> # test & see what happens
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org