[GitHub] [hadoop] steveloughran commented on a diff in pull request #5689: HADOOP-18752. Change fs.s3a.directory.marker.retention to "keep"

["steveloughran (via GitHub)" <gi...@apache.org>]

steveloughran commented on code in PR #5689:
URL: https://github.com/apache/hadoop/pull/5689#discussion_r1214526572


##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/directory_markers.md:
##########
@@ -12,35 +12,40 @@
   limitations under the License. See accompanying LICENSE file.
 -->
 
-# Experimental: Controlling the S3A Directory Marker Behavior
+# Controlling the S3A Directory Marker Behavior
 
-This document discusses an experimental feature of the S3A
-connector since Hadoop 3.3.1: the ability to retain directory
-marker objects above paths containing files or subdirectories.
+This document discusses an performance feature of the S3A
+connector: directory markers are not deleted unless the
+client is explicitly configured to do so.
 
 ## <a name="compatibility"></a> Critical: this is not backwards compatible!
 
 This document shows how the performance of S3 I/O, especially applications
 creating many files (for example Apache Hive) or working with versioned S3 buckets can
 increase performance by changing the S3A directory marker retention policy.
 
-Changing the policy from the default value, `"delete"` _is not backwards compatible_.
+The default policy in this release of hadoop is "keep", 
+which _is not backwards compatible_ with hadoop versions
+released before 2021.
 
-Versions of Hadoop which are incompatible with other marker retention policies,
-as of August 2020.
+The compatibility table of older releases is as follows:
 
-|  Branch    | Compatible Since | Supported           |
-|------------|------------------|---------------------|
-| Hadoop 2.x |       n/a        | WONTFIX             |
-| Hadoop 3.0 |      check       | Read-only           |
-| Hadoop 3.1 |      check       | Read-only           |
-| Hadoop 3.2 |      check       | Read-only           |
-| Hadoop 3.3 |      3.3.1       | Done                |
+| Branch     | Compatible Since | Supported | Released |
+|------------|------------------|-----------|----------|
+| Hadoop 2.x | 2.10.2           | Read-only | 05/2022  |
+| Hadoop 3.0 | n/a              | WONTFIX   |          |
+| Hadoop 3.1 | n/a              | WONTFIX   |          |
+| Hadoop 3.2 | 3.2.2            | Read-only | 01/2022  |
+| Hadoop 3.3 | 3.3.1            | Done      | 01/2021  |

Review Comment:
   yeah, there was some format error already fixed. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org