You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by rm...@apache.org on 2020/05/08 17:00:36 UTC

[flink] branch master updated: [hotfix][docs][s3] Clarify wording of S3 Filesystem support

This is an automated email from the ASF dual-hosted git repository.

rmetzger pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/flink.git


The following commit(s) were added to refs/heads/master by this push:
     new 7c5ac35  [hotfix][docs][s3] Clarify wording of S3 Filesystem support
7c5ac35 is described below

commit 7c5ac3584e42a0e7ebc5e78c532887bf4d383d9d
Author: Robert Metzger <rm...@apache.org>
AuthorDate: Fri May 8 17:41:09 2020 +0200

    [hotfix][docs][s3] Clarify wording of S3 Filesystem support
    
    This closes #12043
---
 docs/ops/filesystems/s3.md | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/docs/ops/filesystems/s3.md b/docs/ops/filesystems/s3.md
index 7c47f1f..feb0b40 100644
--- a/docs/ops/filesystems/s3.md
+++ b/docs/ops/filesystems/s3.md
@@ -60,10 +60,15 @@ Flink provides two file systems to talk to Amazon S3, `flink-s3-fs-presto` and `
 Both implementations are self-contained with no dependency footprint, so there is no need to add Hadoop to the classpath to use them.
 
   - `flink-s3-fs-presto`, registered under the scheme *s3://* and *s3p://*, is based on code from the [Presto project](https://prestodb.io/).
-  You can configure it the same way you can [configure the Presto file system](https://prestodb.io/docs/0.187/connector/hive.html#amazon-s3-configuration) by placing adding the configurations to your `flink-conf.yaml`. Presto is the recommended file system for checkpointing to S3.
+  You can configure it using [the same configuration keys as the Presto file system](https://prestodb.io/docs/0.187/connector/hive.html#amazon-s3-configuration), by adding the configurations to your `flink-conf.yaml`. The Presto S3 implementation is the recommended file system for checkpointing to S3.
 
   - `flink-s3-fs-hadoop`, registered under *s3://* and *s3a://*, based on code from the [Hadoop Project](https://hadoop.apache.org/).
-  The file system can be [configured exactly like Hadoop's s3a](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A) by placing adding the configurations to your `flink-conf.yaml`. It is the only S3 file system with support for the [StreamingFileSink]({{ site.baseurl}}/dev/connectors/streamfile_sink.html).
+  The file system can be [configured using Hadoop's s3a configuration keys](https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html#S3A) by adding the configurations to your `flink-conf.yaml`. 
+  
+     For example, Hadoop has a `fs.s3a.connection.maximum` configuration key. If you want to change it, you need to put `s3.connection.maximum: xyz` to the `flink-conf.yaml`. Flink will internally translate this back to `fs.s3a.connection.maximum`. There is no need to pass configuration parameters using Hadoop's XML configuration files.
+  
+    It is the only S3 file system with support for the [StreamingFileSink]({{ site.baseurl}}/dev/connectors/streamfile_sink.html).
+  
 
 Both `flink-s3-fs-hadoop` and `flink-s3-fs-presto` register default FileSystem
 wrappers for URIs with the *s3://* scheme, `flink-s3-fs-hadoop` also registers
@@ -111,7 +116,7 @@ s3.endpoint: your-endpoint-hostname
 
 ## Configure Path Style Access
 
-Some of the S3 compliant object stores might not have virtual host style addressing enabled by default. In such cases, you will have to provide the property to enable path style access in `flink-conf.yaml`.
+Some S3 compliant object stores might not have virtual host style addressing enabled by default. In such cases, you will have to provide the property to enable path style access in `flink-conf.yaml`.
 
 {% highlight yaml %}
 s3.path.style.access: true