You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/10/07 05:22:18 UTC
[GitHub] [hadoop] mehakmeet commented on a diff in pull request #4478: HADOOP-18304. Improve user-facing S3A committers documentation
mehakmeet commented on code in PR #4478:
URL: https://github.com/apache/hadoop/pull/4478#discussion_r989684378
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committers.md:
##########
@@ -356,19 +351,20 @@ task commit.
However, it has extra requirements of the filesystem
-1. [Obsolete] It requires a consistent object store.
+1. The object store must be consistent.
1. The S3A client must be configured to recognize interactions
-with the magic directories and treat them specially.
+with the magic directories and treat them as a special case.
-Now that Amazon S3 is consistent, the magic committer is enabled by default.
+Now that [Amazon S3 is consistent](https://aws.amazon.com/s3/consistency/),
+the magic committer is enabled by default.
Review Comment:
File committer is still the default committer, right? So what does magic committer being "enabled by default" here mean?
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committers.md:
##########
@@ -530,18 +527,22 @@ performance.
Review Comment:
nit: blank lines
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/committers.md:
##########
@@ -180,20 +180,20 @@ and restarting the job.
whose output is in the job attempt directory, *and only rerunning all uncommitted tasks*.
-This algorithm does not works safely or swiftly with AWS S3 storage because
-tenames go from being fast, atomic operations to slow operations which can fail partway through.
+This algorithm does not work safely or swiftly with AWS S3 storage because
+renames go from being fast, atomic operations to slow operations which can fail partway through.
This then is the problem which the S3A committers address:
-*How to safely and reliably commit work to Amazon S3 or compatible object store*
+>*How to safely and reliably commit work to Amazon S3 or compatible object store.*
Review Comment:
What are we quoting here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org