You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by do...@apache.org on 2022/04/21 03:57:18 UTC
[incubator-inlong-website] branch master updated: [INLONG-361][Doc] Add Kafka/Iceberg/Hive load nodes guide (#362)
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 1015b87c7 [INLONG-361][Doc] Add Kafka/Iceberg/Hive load nodes guide (#362)
1015b87c7 is described below
commit 1015b87c768358b405797c793629daafc86031db
Author: dockerzhang <do...@apache.org>
AuthorDate: Thu Apr 21 11:57:13 2022 +0800
[INLONG-361][Doc] Add Kafka/Iceberg/Hive load nodes guide (#362)
---
docs/data_node/extract_node/kafka.md | 2 +-
docs/data_node/load_node/auto_consumption.md | 9 +++++++++
docs/data_node/load_node/hive.md | 9 +++++++++
docs/data_node/load_node/iceberg.md | 12 ++++++++++++
docs/data_node/load_node/img/hive.png | Bin 0 -> 66070 bytes
docs/data_node/load_node/img/iceberg.png | Bin 0 -> 51252 bytes
docs/data_node/load_node/img/kafka.png | Bin 0 -> 31565 bytes
docs/data_node/load_node/kafka.md | 9 +++++++++
.../current/data_node/extract_node/kafka.md | 3 ++-
.../current/data_node/load_node/auto_consumption.md | 8 ++++++++
.../current/data_node/load_node/hive.md | 9 +++++++++
.../current/data_node/load_node/iceberg.md | 12 ++++++++++++
.../current/data_node/load_node/img/hive.png | Bin 0 -> 50097 bytes
.../current/data_node/load_node/img/iceberg.png | Bin 0 -> 39782 bytes
.../current/data_node/load_node/img/kafka.png | Bin 0 -> 25680 bytes
.../current/data_node/load_node/kafka.md | 9 +++++++++
16 files changed, 80 insertions(+), 2 deletions(-)
diff --git a/docs/data_node/extract_node/kafka.md b/docs/data_node/extract_node/kafka.md
index e8ccdd531..9df2e534b 100644
--- a/docs/data_node/extract_node/kafka.md
+++ b/docs/data_node/extract_node/kafka.md
@@ -5,4 +5,4 @@ sidebar_position: 3
## Configuration
the Dashboard has not supported extracting data from Kafka for this version,
-you can create Kafka data streams from the background via the [managerctl](user_guide/command_line_tools.md).
\ No newline at end of file
+you can create Kafka data streams from the background via the [Command-line Tools](user_guide/command_line_tools.md).
\ No newline at end of file
diff --git a/docs/data_node/load_node/auto_consumption.md b/docs/data_node/load_node/auto_consumption.md
new file mode 100644
index 000000000..ba2e6d6ac
--- /dev/null
+++ b/docs/data_node/load_node/auto_consumption.md
@@ -0,0 +1,9 @@
+---
+title: Auto Push
+sidebar_position: 1
+---
+
+## Overview
+**Auto Consumption** meanings receive data from Message Queue Services (TubeMQ or Pulsar) directly, you can consume the message from MQ
+by [Pulsar SDK Client](https://pulsar.apache.org/docs/en/2.8.3/client-libraries/) or [TubeMQ SDK Client](modules/tubemq/clients_java.md),
+after that, you have to [Parse the InLongMsg](development/inlong_msg.md) to get raw data for forward processing.
\ No newline at end of file
diff --git a/docs/data_node/load_node/hive.md b/docs/data_node/load_node/hive.md
new file mode 100644
index 000000000..5e86a18f9
--- /dev/null
+++ b/docs/data_node/load_node/hive.md
@@ -0,0 +1,9 @@
+---
+title: Hive
+sidebar_position: 2
+---
+
+## Configuration
+When creating a data flow, select `Hive` for the data stream direction, and click "Add" to configure it.
+
+
\ No newline at end of file
diff --git a/docs/data_node/load_node/iceberg.md b/docs/data_node/load_node/iceberg.md
new file mode 100644
index 000000000..da5565727
--- /dev/null
+++ b/docs/data_node/load_node/iceberg.md
@@ -0,0 +1,12 @@
+---
+title: Iceberg
+sidebar_position: 3
+---
+
+## Overview
+[Apache Iceberg](https://iceberg.apache.org/) is a high-performance format for huge analytic tables.
+
+## Configuration
+When creating a data flow, select `Iceberg` for the data stream direction, and click "Add" to configure it.
+
+
\ No newline at end of file
diff --git a/docs/data_node/load_node/img/hive.png b/docs/data_node/load_node/img/hive.png
new file mode 100644
index 000000000..b716d5fda
Binary files /dev/null and b/docs/data_node/load_node/img/hive.png differ
diff --git a/docs/data_node/load_node/img/iceberg.png b/docs/data_node/load_node/img/iceberg.png
new file mode 100644
index 000000000..169812026
Binary files /dev/null and b/docs/data_node/load_node/img/iceberg.png differ
diff --git a/docs/data_node/load_node/img/kafka.png b/docs/data_node/load_node/img/kafka.png
new file mode 100644
index 000000000..ac6f86f95
Binary files /dev/null and b/docs/data_node/load_node/img/kafka.png differ
diff --git a/docs/data_node/load_node/kafka.md b/docs/data_node/load_node/kafka.md
new file mode 100644
index 000000000..6438a4fac
--- /dev/null
+++ b/docs/data_node/load_node/kafka.md
@@ -0,0 +1,9 @@
+---
+title: Kafka
+sidebar_position: 4
+---
+
+## Configuration
+When creating a data flow, select `Kafka` for the data stream direction, and click "Add" to configure it.
+
+
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/kafka.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/kafka.md
index d0ddd7a74..1df9d3351 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/kafka.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/extract_node/kafka.md
@@ -4,4 +4,5 @@ sidebar_position: 3
---
## 配置
-当前版本 Dashboard 还不支持从 Kafka 中抽取数据，你可以通过 [managerctl](user_guide/command_line_tools.md) 命令行工具从后台创建 Kafka 数据流。
\ No newline at end of file
+当前版本 Dashboard 还不支持从 Kafka 中抽取数据，
+你可以通过 [Command-line Tools](user_guide/command_line_tools.md) 命令行工具从后台创建 Kafka 数据流。
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/auto_consumption.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/auto_consumption.md
new file mode 100644
index 000000000..b4488278f
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/auto_consumption.md
@@ -0,0 +1,8 @@
+---
+title: Auto Push
+sidebar_position: 1
+---
+
+## Overview
+**自主消费** 是指直接从消息队列服务中 (TubeMQ or Pulsar) 消费数据, 你可以使用 [Pulsar SDK Client](https://pulsar.apache.org/docs/en/2.8.3/client-libraries/) 或者 [TubeMQ SDK Client](modules/tubemq/clients_java.md) 进行消费，
+获取到数据后，需要通过 [解析 InLongMsg](development/inlong_msg.md) 获取原数据进行下一步处理。
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/hive.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/hive.md
new file mode 100644
index 000000000..874c5030e
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/hive.md
@@ -0,0 +1,9 @@
+---
+title: Hive
+sidebar_position: 2
+---
+
+## 配置
+创建数据流时，数据流向选择 `Hive`，并点击 ”添加“ 进行配置。
+
+
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/iceberg.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/iceberg.md
new file mode 100644
index 000000000..b0825b1fc
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/iceberg.md
@@ -0,0 +1,12 @@
+---
+title: Iceberg
+sidebar_position: 3
+---
+
+## 总览
+[Apache Iceberg](https://iceberg.apache.org/) 是一种用于跟踪超大规模表的新格式。
+
+## 配置
+创建数据流时，数据流向选择 `Iceberg`，并点击 ”添加“ 进行配置。
+
+
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/hive.png b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/hive.png
new file mode 100644
index 000000000..eeddb25b8
Binary files /dev/null and b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/hive.png differ
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/iceberg.png b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/iceberg.png
new file mode 100644
index 000000000..c162acfcd
Binary files /dev/null and b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/iceberg.png differ
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/kafka.png b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/kafka.png
new file mode 100644
index 000000000..dc119deb8
Binary files /dev/null and b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/img/kafka.png differ
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/kafka.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/kafka.md
new file mode 100644
index 000000000..56787dfb6
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/data_node/load_node/kafka.md
@@ -0,0 +1,9 @@
+---
+title: Kafka
+sidebar_position: 4
+---
+
+## Configuration
+创建数据流时，数据流向选择 `Kafka`，并点击 ”添加“ 进行配置。
+
+
\ No newline at end of file