You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Steve Loughran (JIRA)" <ji...@apache.org> on 2017/09/07 13:39:00 UTC
[jira] [Comment Edited] (HADOOP-14738) Remove S3N and obsolete bits
of S3A; rework docs
[ https://issues.apache.org/jira/browse/HADOOP-14738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16156947#comment-16156947 ]
Steve Loughran edited comment on HADOOP-14738 at 9/7/17 1:38 PM:
-----------------------------------------------------------------
Patch 004; also remove the old S3AOutputStream,
Cutting the classic S3A output stream removes another scale test, implicitly increases test coverage, and simplifies the doc. I've retained a check for the option in the S3A initialize() method, but all it does it warn the user that the option is gone.
Docs all reviewed. cross-doc links reviewed, etc.
Tested: s3 ireland, with and without s3guard; at scale
was (Author: stevel@apache.org):
Patch 004; also remove the old S3AOutputStream,
Cutting the classic S3A output stream removes another scale test (!), implicitly increases test coverage, and simplifies the doc. I've retained a check for the option in the S3A initialize() method, but all it does it warn the user that the option is gone.
Docs all reviewed. cross-doc links reviewed, etc.
Tested: s3 ireland, with and without s3guard; at scale
> Remove S3N and obsolete bits of S3A; rework docs
> ------------------------------------------------
>
> Key: HADOOP-14738
> URL: https://issues.apache.org/jira/browse/HADOOP-14738
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 2.9.0, 3.0.0-beta1
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
> Attachments: HADOOP-14738-002.patch, HADOOP-14738-003.patch, HADOOP-14738-004.patch, HADOOP-14739-001.patch
>
>
> We are all happy with S3A; it's been stable since Hadoop 2.7 and high-perf since Hadoop 2.8
> It's now time to kill S3N off, remove the source, the tests, the transitive dependencies. This patch does that.
> It also removes the obsolete, original s3a output stream; the fast/block upload stream has been stable and is much more manageable and maintained (put differently: we don't ever look at the original S3A output stream, and tell people not to use it for performance reasons).
> As well as cutting the features, this patch updates the aws docs with
> * split out s3n migration page
> * split out troubleshooting page
> * rework of the "uploading data with s3a" section of index.md, as there's no need to discuss the slow upload except in the past tense...all that is needed is to list the buffering and thread tuning options of the block uploader.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org