You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by tw...@apache.org on 2021/10/18 18:27:26 UTC

[flink] branch release-1.14 updated (c337fff -> 7ed5ea6)

This is an automated email from the ASF dual-hosted git repository.

twalthr pushed a change to branch release-1.14
in repository https://gitbox.apache.org/repos/asf/flink.git.


    from c337fff  [FLINK-24465] Fix disabling buffer timeout for pipelined exchanges
     new 663734f  [hotfix][docs] Add additional sentence for minio users
     new 702a2e3  [hotfix][docs] Specify that json format works for append only streams
     new 7ed5ea6  [hotfix][docs] Specify that there is no defined ingestion order for the files in a directory when using the filesystem connector

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 docs/content/docs/connectors/table/filesystem.md   | 6 ++++++
 docs/content/docs/connectors/table/formats/json.md | 2 ++
 docs/content/docs/deployment/filesystems/s3.md     | 4 ++--
 3 files changed, 10 insertions(+), 2 deletions(-)

[flink] 01/03: [hotfix][docs] Add additional sentence for minio users

Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

twalthr pushed a commit to branch release-1.14
in repository https://gitbox.apache.org/repos/asf/flink.git

commit 663734fec8bb231bf87c2280b59b0b455247efed
Author: slinkydeveloper <fr...@gmail.com>
AuthorDate: Thu Sep 23 15:33:53 2021 +0200

    [hotfix][docs] Add additional sentence for minio users
    
    Signed-off-by: slinkydeveloper <fr...@gmail.com>
---
 docs/content/docs/deployment/filesystems/s3.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/docs/content/docs/deployment/filesystems/s3.md b/docs/content/docs/deployment/filesystems/s3.md
index d09ffdc..ac5211f 100644
--- a/docs/content/docs/deployment/filesystems/s3.md
+++ b/docs/content/docs/deployment/filesystems/s3.md
@@ -111,7 +111,7 @@ s3.secret-key: your-secret-key
 
 ## Configure Non-S3 Endpoint
 
-The S3 Filesystems also support using S3 compliant object stores such as [IBM's Cloud Object Storage](https://www.ibm.com/cloud/object-storage) and [Minio](https://min.io/).
+The S3 Filesystems also support using S3 compliant object stores such as [IBM's Cloud Object Storage](https://www.ibm.com/cloud/object-storage) and [MinIO](https://min.io/).
 To do so, configure your endpoint in `flink-conf.yaml`.
 
 ```yaml
@@ -120,7 +120,7 @@ s3.endpoint: your-endpoint-hostname
 
 ## Configure Path Style Access
 
-Some S3 compliant object stores might not have virtual host style addressing enabled by default. In such cases, you will have to provide the property to enable path style access in `flink-conf.yaml`.
+Some S3 compliant object stores might not have virtual host style addressing enabled by default, for example when using Standalone MinIO for testing purpose. In such cases, you will have to provide the property to enable path style access in `flink-conf.yaml`.
 
 ```yaml
 s3.path.style.access: true

[flink] 02/03: [hotfix][docs] Specify that json format works for append only streams

Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

twalthr pushed a commit to branch release-1.14
in repository https://gitbox.apache.org/repos/asf/flink.git

commit 702a2e3cbc096dd4396cbaa9e228eb05fe2956a5
Author: slinkydeveloper <fr...@gmail.com>
AuthorDate: Thu Sep 23 16:24:37 2021 +0200

    [hotfix][docs] Specify that json format works for append only streams
    
    Signed-off-by: slinkydeveloper <fr...@gmail.com>
---
 docs/content/docs/connectors/table/formats/json.md | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/docs/content/docs/connectors/table/formats/json.md b/docs/content/docs/connectors/table/formats/json.md
index c61e28b..56b8c84 100644
--- a/docs/content/docs/connectors/table/formats/json.md
+++ b/docs/content/docs/connectors/table/formats/json.md
@@ -31,6 +31,8 @@ under the License.
 
 The [JSON](https://www.json.org/json-en.html) format allows to read and write JSON data based on an JSON schema. Currently, the JSON schema is derived from table schema.
 
+The JSON format supports append-only streams, unless you're using a connector that explicitly support retract streams and/or upsert streams like the [Upsert Kafka]({{< ref "docs/connectors/table/upsert-kafka" >}}) connector. If you need to write retract streams and/or upsert streams, we suggest you to look at CDC JSON formats like [Debezium JSON]({{< ref "docs/connectors/table/formats/debezium" >}}) and [Canal JSON]({{< ref "docs/connectors/table/formats/canal" >}}).
+
 Dependencies
 ------------

[flink] 03/03: [hotfix][docs] Specify that there is no defined ingestion order for the files in a directory when using the filesystem connector

Posted by tw...@apache.org.
This is an automated email from the ASF dual-hosted git repository.

twalthr pushed a commit to branch release-1.14
in repository https://gitbox.apache.org/repos/asf/flink.git

commit 7ed5ea65845bf3be81ac9115988d1610e7d1ea5e
Author: slinkydeveloper <fr...@gmail.com>
AuthorDate: Thu Sep 23 16:32:50 2021 +0200

    [hotfix][docs] Specify that there is no defined ingestion order for the files in a directory when using the filesystem connector
    
    Signed-off-by: slinkydeveloper <fr...@gmail.com>
    
    This closes #17343.
---
 docs/content/docs/connectors/table/filesystem.md | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/docs/content/docs/connectors/table/filesystem.md b/docs/content/docs/connectors/table/filesystem.md
index 4fa3458..7f143b3 100644
--- a/docs/content/docs/connectors/table/filesystem.md
+++ b/docs/content/docs/connectors/table/filesystem.md
@@ -102,6 +102,12 @@ The file system connector supports multiple formats:
  - Canal-JSON: [canal-json]({{< ref "docs/connectors/table/formats/canal" >}}).
  - Raw: [raw]({{< ref "docs/connectors/table/formats/raw" >}}).
 
+## Source
+
+The file system connector can be used to read single files or entire directories into a single table.
+
+When using a directory as the source path, there is **no defined order of ingestion** for the files inside the directory.
+
 ## Streaming Sink
 
 The file system connector supports streaming writes, based on Flink's [Streaming File Sink]({{< ref "docs/connectors/datastream/streamfile_sink" >}})