You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2018/04/27 10:24:00 UTC

[jira] [Created] (HADOOP-15421) Stabilise/formalise the JSON _SUCCESS format used in the S3A committers

Steve Loughran created HADOOP-15421:
---------------------------------------

             Summary: Stabilise/formalise the JSON _SUCCESS format used in the S3A committers
                 Key: HADOOP-15421
                 URL: https://issues.apache.org/jira/browse/HADOOP-15421
             Project: Hadoop Common
          Issue Type: Sub-task
    Affects Versions: 3.2.0
            Reporter: Steve Loughran


the S3A committers rely on an atomic PUT to save a JSON summary of the job to the dest FS, containing files, statistics, etc. This is for internal testing, but it turns out to be useful for spark integration testing, Hive, etc.

IBM's stocator also generated a manifest.

Proposed: come up with (an extensible) design that we are happy with as a long lived format.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org