You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by do...@apache.org on 2022/11/15 14:15:27 UTC
[inlong-website] 01/01: [INLONG-600][Doc] Update the concept and introduction guide
This is an automated email from the ASF dual-hosted git repository.
dockerzhang pushed a commit to branch fix-600
in repository https://gitbox.apache.org/repos/asf/inlong-website.git
commit aa1f59dcb3cc1fb979ea4bcf123f79f553faf710
Author: dockerzhang <do...@apache.org>
AuthorDate: Tue Nov 15 22:15:15 2022 +0800
[INLONG-600][Doc] Update the concept and introduction guide
---
docs/design_and_concept/basic_concept.md | 28 ++++++++++++----------
docs/introduction.md | 6 ++---
.../current/design_and_concept/basic_concept.md | 28 ++++++++++++----------
.../current/introduction.md | 12 +++++-----
4 files changed, 39 insertions(+), 35 deletions(-)
diff --git a/docs/design_and_concept/basic_concept.md b/docs/design_and_concept/basic_concept.md
index 823e96dee4..cf41f67583 100644
--- a/docs/design_and_concept/basic_concept.md
+++ b/docs/design_and_concept/basic_concept.md
@@ -3,16 +3,18 @@ title: Basic Concept
sidebar_position: 1
---
-| Name | Description | Other |
-|--------------------------|----------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
-| Standard Architecture | Contains all InLong components such as InLong Agent/Manager/MQ/Sort/Dashboard | Suitable for massive data and large-scale production environments |
-| Lightweight Architecture | Contains only one component of InLong Sort, which can be used with Manager/Dashboard | The lightweight architecture is simple and flexible, suitable for small-scale data |
-| Group | Data Streams Group, it contains multiple data streams, and one Group represents one data ingestion. | Group has attributes such as ID and Name. |
-| Stream | Data Stream, a stream has a specific flow direction. | Stream has attributes such as ID, Name, and data fields. |
-| Node | Data Node, including `Extract Node` and `Load Node`, stands for the data source and sink types separately. | |
-| InLongMsg | InLong data format, if you consume message directly from the message queue, you need to perform `InLongMsg` parsing first. | |
-| Agent | The standard architecture uses Agent for data collection, and Agent represents different types of collection capabilities. | It contains File Agent, SQL Agent, Binlog Agent, etc. |
-| DataProxy | Forward received data to different message queues. | Supports data transmission blocking, placing retransmission. |
-| Sort | Data stream sorting. | Sort-flink based on Flink, sort-standalone for local sorting. |
-| TubeMQ | InLong's self-developed message queuing service | It can also be called Tube, with low-cost, high-performance features. |
-| Pulsar | [Apache Pulsar](https://pulsar.apache.org/), a high-performance, high-consistency message queue service | |
\ No newline at end of file
+| Name | Description | Other |
+|-------------|----------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------|
+| Standard | Standard Architecture, contains all InLong components such as InLong Agent, Manager, MQ, Sort, Dashboard | Suitable for massive data and large-scale production environments |
+| Lightweight | Lightweight Architecture, contains only one component of InLong Sort, which also can be used with Manager, Dashboard | The lightweight architecture is simple and flexible, suitable for small-scale data |
+| Group | Data Streams Group, it contains multiple data streams, and one Group represents one data business unit. | Group has attributes such as ID and Name. |
+| Stream | Data Stream, a stream has a specific data source, data format and data sink. | Stream has attributes such as ID, Name, and data fields. |
+| Node | Data Node, including `Extract Node` and `Load Node`, stands for the data source and sink types separately. | |
+| InLongMsg | InLong data format, if you consume message directly from the message queue, you need to perform `InLongMsg` parsing first. | |
+| Cluster | Each component can form a single cluster. | Contains cluster name, label, necessary information for each component, etc. |
+| Tag | Clusters of different components can use the same tag to represent a set of data stream execution units. | Currently tag are only available for clusters |
+| Agent | The standard architecture uses Agent for data collection, and Agent represents different types of collection capabilities. | It contains File Agent, SQL Agent, Binlog Agent, etc. |
+| DataProxy | Forward received data to different message queues. | Supports data transmission blocking, placing retransmission. |
+| Sort | Data stream sorting. | Sort-flink based on Flink, sort-standalone for local sorting. |
+| TubeMQ | InLong's self-developed message queuing service | It can also be called Tube, with low-cost, high-performance features. |
+| Pulsar | [Apache Pulsar](https://pulsar.apache.org/), a high-performance, high-consistency message queue service | |
\ No newline at end of file
diff --git a/docs/introduction.md b/docs/introduction.md
index 685e1ef1a9..77699c6d8f 100644
--- a/docs/introduction.md
+++ b/docs/introduction.md
@@ -1,5 +1,5 @@
---
-title: InLong Introduction
+title: Introduction
sidebar_position: 1
---
@@ -45,10 +45,10 @@ values={[
{label: 'Standard', value: 'standard'},
{label: 'Lightweight', value: 'lightweight'},
]}>
-<TabItem value="standard">
+<TabItem value="standard"> Standard Architecture: contains all InLong components such as InLong Agent, Manager, MQ, Sort, Dashboard, which is suitable for massive data and large-scale production environments.
<img src="/img/inlong-structure-en.png" align="center" alt="Apache InLong"/>
</TabItem>
-<TabItem value="lightweight">
+<TabItem value="lightweight"> Lightweight Architecture: contains only one component of InLong Sort, which also can be used with Manager, Dashboard, and it is simple and flexible, suitable for small-scale data.
<img src="/img/inlong-structure-light.png" align="center" style={{objectFit: 'cover', height: '460px', width: '860px'}} alt="Apache InLong"/>
</TabItem>
</Tabs>
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/design_and_concept/basic_concept.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/design_and_concept/basic_concept.md
index 3195b2850a..2f7e1733d2 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/design_and_concept/basic_concept.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/design_and_concept/basic_concept.md
@@ -3,16 +3,18 @@ title: 基本概念
sidebar_position: 1
---
-| Name | Description | Other |
-|---------------------------|--------------------------------------------------------------|-----------------------------------------------|
-| Standard Architecture | 标准架构，包含 InLong Agent/Manager/MQ/Sort/Dashboard 等所有 InLong 组件 | 适合海量数据、大规模生产环境 |
-| Lightweight Architecture | 轻量化架构，只包含 InLong Sort 一个组件，可以搭配 Manager/Dashboard 一起使用 | 轻量化架构简单、灵活，适合小规模数据 |
-| Group | 数据流组，包含多个数据流，一个Group 代表一个数据接入 | Group 有ID、Name 等属性 |
-| Stream | 数据流，一个数据流有具体的流向 | Stream 有ID、Name、数据字段等属性 |
-| Node | 数据节点，包括`Extract Node` 和 `Load Node`，分别代表数据源类型和数据流向目标类型 | |
-| InLongMsg | InLong 数据格式，如果从消息队列中直接消费，需要先进行 `InLongMsg` 解析 | |
-| Agent | 标准架构使用 Agent 进行数据采集，Agent 代表不同类型的采集能力 | 包含文件 Agent、SQL Agent、Binlog Agent 等 |
-| DataProxy | 将接收到的数据转发到不同的消息队列 | 支持数据发送阻塞和落盘重发 |
-| Sort | 数据流分拣 | 主要有基于 Flink 的 sort-flink，sort-standalone 本地分拣 |
-| TubeMQ | InLong 自带的消息队列服务 | 也可以叫 Tube，拥有低成本、高性能特性 |
-| Pulsar | 即[Apache Pulsar](https://pulsar.apache.org/), 高性能、高一致性消息队列服务 | |
\ No newline at end of file
+| Name | Description | Other |
+|-------------|--------------------------------------------------------------|-----------------------------------------------|
+| Standard | 标准架构，包含 InLong Agent、Manager、MQ、Sort、Dashboard 等所有 InLong 组件 | 适合海量数据、大规模生产环境 |
+| Lightweight | 轻量化架构，只包含 InLong Sort 一个组件，也可以搭配 Manager，Dashboard 一起使用 | 轻量化架构简单、灵活，适合小规模数据 |
+| Group | 数据流组，包含多个数据流，一个 Group 代表一个数据业务单元 | Group 有 ID、Name 等属性 |
+| Stream | 数据流，一个数据流有具体的数据源、数据格式和数据流向 | Stream 有 ID、Name、数据字段等属性 |
+| Node | 数据节点，包括 `Extract Node` 和 `Load Node`，分别代表数据源类型和数据流向目标类型 | |
+| InLongMsg | InLong 数据格式，如果从消息队列中直接消费，需要先进行 `InLongMsg` 解析 | |
+| Cluster | 集群，每个组件都可以构成单一集群。 | 包含集群名、标签，每个组件必要信息等 |
+| Tag | 标签，不同组件的集群可以使用相同标签，代表一组数据流执行单元 | 目前标签只适用于集群 |
+| Agent | 标准架构使用 Agent 进行数据采集，Agent 代表不同类型的采集能力 | 包含文件 Agent、SQL Agent、Binlog Agent 等 |
+| DataProxy | 将接收到的数据转发到不同的消息队列 | 支持数据发送阻塞和落盘重发 |
+| Sort | 数据流分拣 | 主要有基于 Flink 的 sort-flink，sort-standalone 本地分拣 |
+| TubeMQ | InLong 自带的消息队列服务 | 也可以叫 Tube，拥有低成本、高性能特性 |
+| Pulsar | 即 [Apache Pulsar](https://pulsar.apache.org/)，高性能、高一致性消息队列服务 | |
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/introduction.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/introduction.md
index 2db4d30874..b452256aa2 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/introduction.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/introduction.md
@@ -1,5 +1,5 @@
---
-title: InLong 简介
+title: 简介
sidebar_position: 1
---
@@ -39,13 +39,13 @@ Apache InLong 依托 10 万亿级别的数据接入和处理能力，整合了
<Tabs
defaultValue="standard"
values={[
-{label: 'Standard', value: 'standard'},
-{label: 'Lightweight', value: 'lightweight'},
+{label: '标准', value: 'standard'},
+{label: '轻量化', value: 'lightweight'},
]}>
-<TabItem value="standard">
+<TabItem value="standard"> 标准架构：包含 InLong Agent、Manager、MQ、Sort、Dashboard 等所有 InLong 组件，适合海量数据、大规模生产环境。
<img src="/img/inlong-structure-zh.png" align="center" alt="Apache InLong"/>
</TabItem>
-<TabItem value="lightweight">
+<TabItem value="lightweight"> 轻量化架构：只包含 InLong Sort 一个组件，也可以搭配 Manager，Dashboard 一起使用。轻量化架构简单、灵活，适合小规模数据。
<img src="/img/inlong-structure-light.png" align="center" style={{objectFit: 'cover', height: '460px', width: '860px'}} alt="Apache InLong"/>
</TabItem>
</Tabs>
@@ -53,7 +53,7 @@ values={[
## 模块
Apache InLong 服务于数据采集到落地的整个生命周期，按数据的不同阶段提供不同的处理模块，主要包括：
-- **inlong-agent**，数据采集服务，包括文件采集、DB采集等。
+- **inlong-agent**，数据采集服务，包括文件采集、DB 采集等。
- **inlong-dataproxy**，一个基于 Flume-ng 的 Proxy 组件，支持数据发送阻塞和落盘重发，拥有将接收到的数据转发到不同 MQ（消息队列）的能力。
- **inlong-tubemq**，腾讯自研的消息队列服务，专注于大数据场景下海量数据的高性能存储和传输，在海量实践和低成本方面有着良好的核心优势。
- **inlong-sort**，对从不同的 MQ 消费到的数据进行 ETL 处理，然后汇聚并写入 Hive、ClickHouse、Hbase、Iceberg 等存储系统。