You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-dev@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2017/12/06 13:04:01 UTC

[jira] [Resolved] (HADOOP-13968) S3a FS to support "__magic" path for the special "unmaterialized" writes

     [ https://issues.apache.org/jira/browse/HADOOP-13968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Steve Loughran resolved HADOOP-13968.
-------------------------------------
       Resolution: Duplicate
         Assignee: Steve Loughran
    Fix Version/s: 3.1.0

> S3a FS to support "__magic" path for the special "unmaterialized" writes
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-13968
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13968
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0-beta1
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>             Fix For: 3.1.0
>
>
> S3AFileSystem to add support for a special path, such as  {{.temp_pending_put/}} or similar, which, when used as the base of a path, indicates that the file is actually to be saved to the parent dir, but only via a delayed put commit operation.
> At the same time, we may need blocks on some normal fileIO ops under these dirs, especially rename and delete, as this would cause serious problems including data loss and large bills for pending data.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org