You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by zi...@apache.org on 2022/06/16 03:31:52 UTC

[incubator-inlong] branch master updated: [INLONG-4670][Sort] Update the README.md for the Sort module (#4672)

This is an automated email from the ASF dual-hosted git repository.

zirui pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-inlong.git


The following commit(s) were added to refs/heads/master by this push:
     new ea8a215d8 [INLONG-4670][Sort] Update the README.md for the Sort module (#4672)
ea8a215d8 is described below

commit ea8a215d81dc1a775177aad293599bd05fcb6943
Author: yunqingmoswu <44...@users.noreply.github.com>
AuthorDate: Thu Jun 16 11:31:48 2022 +0800

    [INLONG-4670][Sort] Update the README.md for the Sort module (#4672)
---
 inlong-sort/README.md | 68 ++++++++++++++++++++++++++++++++-------------------
 1 file changed, 43 insertions(+), 25 deletions(-)

diff --git a/inlong-sort/README.md b/inlong-sort/README.md
index 98333d863..e870eaca0 100644
--- a/inlong-sort/README.md
+++ b/inlong-sort/README.md
@@ -1,35 +1,53 @@
 # Description
-## overview
-Inlong-sort is used to extract data from different source systems, then transforms the data and finally loads the data into diffrent storage systems.
-Inlong-sort is simply an Flink application, and relys on Inlong-manager to manage meta data(such as the source informations and storage informations)
 
-## features
-### multi-tenancy
-Inlong-sort is an multi-tenancy system, which means you can extract data from different sources(these sources must be of the same source type) and load data into different sinks(these sinks must be of the same storage type).
-e.g. you can extract data form different topics of inlong-tubemq and the load them to different hive clusters.
+# Overview
 
-### change meta data without restart
-Inlong-sort uses zookeeper to manage its meta data, every time you change meta data on zk, inlong-sort application will be informed immediately.
-e.g if you want to change the schema of your data, just change the meta data on zk without restart your inlong-sort application.
+InLong-Sort is used to extract data from different source systems, then transforms the data and finally loads the data
+into diffrent storage systems.
+InLong-Sort is simply a Flink Application, and relys on InLong-Manager to manage meta data(such as the source
+informations and storage informations).
 
-## supported sources
-**inlong-tubemq**
+# Features
 
-## supported storages
-**hive**
-Currently we just support parquet file format in hive
+## Supported Extract Node
 
-**clickhouse**
+- Pulsar
+- MySQL
+- Kafka
+- MongoDB
+- PostgreSQL
+- HDFS
 
-## limitations
-Currently, we just support extracting specified fields in the stage of **Transform**.
+## Supported Transform
 
-# Plans in the future
-## More kinds of source systems
-pulsar, kafka and etc
+- String Split
+- String Regular Replace
+- String Regular Replace First Matched Value
+- Data Filter
+- Data Distinct
+- Regular Join
 
-## More kinds of storage systems
-Hbase, Elastic Search, and etc
+## Supported Load Node
 
-## More kinds of file format in hive sink
-sequence file, orc
+- Hive
+- Kafka
+- HBase
+- ClickHouse
+- Iceberg
+- PostgreSQL
+- HDFS
+- TDSQL Postgres
+
+## Future Plans
+
+### More kinds of Extract Node
+
+Oracle, SqlServer, and etc.
+
+### More kinds of Transform
+
+Time window aggregation, Content extraction, Type conversion, Time format conversion, and etc.
+
+### More kinds of Load Node
+
+Elasticsearch, and etc.