You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by lz...@apache.org on 2023/01/09 06:40:24 UTC
[flink-table-store] branch release-0.3 updated (ed96a7ca -> 651bffbf)
This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a change to branch release-0.3
in repository https://gitbox.apache.org/repos/asf/flink-table-store.git
from ed96a7ca [FLINK-30595] support create table like
new 49cedad2 [hotfix][doc] Document partial-update.ignore-delete
new 651bffbf [hotfix][doc] Add download to oss and s3
The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/content/docs/features/table-types.md | 19 +++++++++++++++++--
docs/content/docs/filesystems/oss.md | 26 ++++++++++++++++++++++++++
docs/content/docs/filesystems/s3.md | 26 ++++++++++++++++++++++++++
3 files changed, 69 insertions(+), 2 deletions(-)
[flink-table-store] 02/02: [hotfix][doc] Add download to oss and s3
Posted by lz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch release-0.3
in repository https://gitbox.apache.org/repos/asf/flink-table-store.git
commit 651bffbfc5c95e7ffb79f3a36c268d6fdb527b76
Author: JingsongLi <lz...@aliyun.com>
AuthorDate: Mon Jan 9 14:39:40 2023 +0800
[hotfix][doc] Add download to oss and s3
---
docs/content/docs/filesystems/oss.md | 26 ++++++++++++++++++++++++++
docs/content/docs/filesystems/s3.md | 26 ++++++++++++++++++++++++++
2 files changed, 52 insertions(+)
diff --git a/docs/content/docs/filesystems/oss.md b/docs/content/docs/filesystems/oss.md
index 1e0bbc0c..a83c26cd 100644
--- a/docs/content/docs/filesystems/oss.md
+++ b/docs/content/docs/filesystems/oss.md
@@ -26,6 +26,17 @@ under the License.
# OSS
+{{< stable >}}
+
+## Download
+
+[Download](https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-store-oss/{{< version >}}/flink-table-store-oss-{{< version >}}.jar)
+flink table store shaded jar for Spark, Hive and Trino.
+
+{{< /stable >}}
+
+{{< unstable >}}
+
## Build
To build from source code, either [download the source of a release](https://flink.apache.org/downloads.html) or [clone the git repository]({{< github_repo >}}).
@@ -39,6 +50,8 @@ mvn clean install -DskipTests
You can find the shaded jars under
`./flink-table-store-filesystems/flink-table-store-oss/target/flink-table-store-oss-{{< version >}}.jar`.
+{{< /unstable >}}
+
## Usage
{{< tabs "oss" >}}
@@ -90,4 +103,17 @@ SELECT COUNT(1) FROM test_table;
{{< /tab >}}
+{{< tab "Trino" >}}
+
+Place `flink-table-store-oss-{{< version >}}.jar` together with `flink-table-store-trino-{{< version >}}.jar` under `plugin/tablestore` directory.
+
+Add options in `etc/catalog/tablestore.properties`.
+```shell
+fs.oss.endpoint=oss-cn-hangzhou.aliyuncs.com
+fs.oss.accessKeyId=xxx
+fs.oss.accessKeySecret=yyy
+```
+
+{{< /tab >}}
+
{{< /tabs >}}
diff --git a/docs/content/docs/filesystems/s3.md b/docs/content/docs/filesystems/s3.md
index 02179960..f89f21b7 100644
--- a/docs/content/docs/filesystems/s3.md
+++ b/docs/content/docs/filesystems/s3.md
@@ -26,6 +26,17 @@ under the License.
# S3
+{{< stable >}}
+
+## Download
+
+[Download](https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-store-s3/{{< version >}}/flink-table-store-s3-{{< version >}}.jar)
+flink table store shaded jar for Spark, Hive and Trino.
+
+{{< /stable >}}
+
+{{< unstable >}}
+
## Build
To build from source code, either [download the source of a release](https://flink.apache.org/downloads.html) or [clone the git repository]({{< github_repo >}}).
@@ -39,6 +50,8 @@ mvn clean install -DskipTests
You can find the shaded jars under
`./flink-table-store-filesystems/flink-table-store-s3/target/flink-table-store-s3-{{< version >}}.jar`.
+{{< /unstable >}}
+
## Usage
{{< tabs "oss" >}}
@@ -90,6 +103,19 @@ SELECT COUNT(1) FROM test_table;
{{< /tab >}}
+{{< tab "Trino" >}}
+
+Place `flink-table-store-s3-{{< version >}}.jar` together with `flink-table-store-trino-{{< version >}}.jar` under `plugin/tablestore` directory.
+
+Add options in `etc/catalog/tablestore.properties`.
+```shell
+s3.endpoint=your-endpoint-hostname
+s3.access-key=xxx
+s3.secret-key=yyy
+```
+
+{{< /tab >}}
+
{{< /tabs >}}
## S3 Complaint Object Stores
[flink-table-store] 01/02: [hotfix][doc] Document partial-update.ignore-delete
Posted by lz...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch release-0.3
in repository https://gitbox.apache.org/repos/asf/flink-table-store.git
commit 49cedad2d29e7448dda334b07c7e0b9d16d8239a
Author: JingsongLi <lz...@aliyun.com>
AuthorDate: Mon Jan 9 14:26:54 2023 +0800
[hotfix][doc] Document partial-update.ignore-delete
---
docs/content/docs/features/table-types.md | 19 +++++++++++++++++--
1 file changed, 17 insertions(+), 2 deletions(-)
diff --git a/docs/content/docs/features/table-types.md b/docs/content/docs/features/table-types.md
index 2826ab44..c2911e66 100644
--- a/docs/content/docs/features/table-types.md
+++ b/docs/content/docs/features/table-types.md
@@ -50,9 +50,20 @@ Specifically, if the latest record is a `DELETE` record, all records with the sa
By specifying `'merge-engine' = 'partial-update'`, users can set columns of a record across multiple updates and finally get a complete record. Specifically, value fields are updated to the latest data one by one under the same primary key, but null values are not overwritten.
-For example, let's say Table Store receives three records `<1, 23.0, 10, NULL>`, `<1, NULL, NULL, 'This is a book'>` and `<1, 25.2, NULL, NULL>`, where the first column is the primary key. The final result will be `<1, 25.2, 10, 'This is a book'>`.
+For example, let's say Table Store receives three records:
+- `<1, 23.0, 10, NULL>`-
+- `<1, NULL, NULL, 'This is a book'>`
+- `<1, 25.2, NULL, NULL>`
-NOTE: For streaming queries, `partial-update` merge engine must be used together with `full-compaction` [changelog producer]({{< ref "docs/features/table-types#changelog-producers" >}}).
+If the first column is the primary key. The final result will be `<1, 25.2, 10, 'This is a book'>`.
+
+{{< hint info >}}
+For streaming queries, `partial-update` merge engine must be used together with `full-compaction` [changelog producer]({{< ref "docs/features/table-types#changelog-producers" >}}).
+{{< /hint >}}
+
+{{< hint info >}}
+Partial cannot receive `DELETE` messages because the behavior cannot be defined. You can configure `partial-update.ignore-delete` to ignore `DELETE` messages.
+{{< /hint >}}
#### Aggregation
@@ -91,6 +102,10 @@ Current supported aggregate functions and data types are:
* `listagg`: supports STRING data type.
* `bool_and` / `bool_or`: support BOOLEAN data type.
+{{< hint info >}}
+For streaming queries, `aggregation` merge engine must be used together with `full-compaction` [changelog producer]({{< ref "docs/features/table-types#changelog-producers" >}}).
+{{< /hint >}}
+
### Changelog Producers
Streaming queries will continuously produce latest changes. These changes can come from the underlying table files or from an [external log system]({{< ref "docs/features/external-log-systems" >}}) like Kafka. Compared to the external log system, changes from table files have lower cost but higher latency (depending on how often snapshots are created).