You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@doris.apache.org by ji...@apache.org on 2022/07/22 10:30:41 UTC
[doris-website] branch master updated: add cloudcanal
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new dbeea78528 add cloudcanal
dbeea78528 is described below
commit dbeea78528fd33e1d2d352b20ce39ec01c21a3b9
Author: jiafeng.zhang <zh...@gmail.com>
AuthorDate: Fri Jul 22 16:59:53 2022 +0800
add cloudcanal
add cloudcanal
---
docs/ecosystem/cloudcanal.md | 35 +++++++
.../current/ecosystem/cloudcanal.md | 116 +++++++++++++++++++++
sidebars.json | 1 +
static/images/cloudcanal/cloudcanal-1.jpg | Bin 0 -> 169852 bytes
static/images/cloudcanal/cloudcanal-2.jpg | Bin 0 -> 369963 bytes
static/images/cloudcanal/cloudcanal-3.jpg | Bin 0 -> 420489 bytes
static/images/cloudcanal/cloudcanal-4.png | Bin 0 -> 239668 bytes
static/images/cloudcanal/cloudcanal-5.png | Bin 0 -> 250224 bytes
static/images/cloudcanal/cloudcanal-6.png | Bin 0 -> 339062 bytes
static/images/cloudcanal/cloudcanal-7.png | Bin 0 -> 357200 bytes
static/images/cloudcanal/cloudcanal-8.jpg | Bin 0 -> 308720 bytes
11 files changed, 152 insertions(+)
diff --git a/docs/ecosystem/cloudcanal.md b/docs/ecosystem/cloudcanal.md
new file mode 100644
index 0000000000..62dc2a87a2
--- /dev/null
+++ b/docs/ecosystem/cloudcanal.md
@@ -0,0 +1,35 @@
+---
+{
+ "title": "CloudCanal Data Import",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# CloudCanal Data Import
+
+## Introduction
+
+CloudCanal Community Edition is a free data migration and synchronization platform published by [ClouGence Company](https://www.clougence.com) that integrates structure migration, full data migration/check/correction, and incremental real-time synchronization. Product contains complete
+Its productization capabilities help enterprises break data silos, complete data integration and interoperability, and make better use of data.
+
+
+> There is no English version of this document, please switch to the Chinese version.
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/cloudcanal.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/cloudcanal.md
new file mode 100644
index 0000000000..a1e7b23228
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/ecosystem/cloudcanal.md
@@ -0,0 +1,116 @@
+---
+{
+ "title": "CloudCanal 数据导入",
+ "language": "zh-CN"
+}
+---
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# CloudCanal 数据导入
+
+## 介绍
+
+CloudCanal 社区版是一款由 [ClouGence 公司](https://www.clougence.com) 发行的集结构迁移、数据全量迁移/校验/订正、增量实时同步为一体的免费数据迁移同步平台。产品包含完整的产品化能力，助力企业打破数据孤岛、完成数据互融互通，从而更好的使用数据。
+
+
+## 下载安装
+
+[CloudCanal 最新版下载地址](https://clougence.com)
+
+[CloudCanal 社区版安装部署参考文档](https://doc-cloudcanal.clougence.com/operation/install_linux)
+
+## 功能说明
+
+- 推荐使用2.2.5.0及以上的CloudCanal版本写入Doris
+- 建议您在使用 CloudCanal 将 **增量数据** 导入至 Doris 时，控制导入的频率，CloudCanal写入Doris的默认导入频率可以通过参数`realFlushPauseSec`调整，默认为10秒。
+- 当前社区版本最大的内存配置为2g，如果同步任务运行产生OOM异常或者GC停顿严重可以调小以下参数减少批次大小从而减少内存占用。全量参数为`fullBatchSize`和`fullRingBufferSize`，增量参数为`increBatchSize`和`increRingBufferSize`
+- 支持的源端以及功能项：
+
+ | 数据源 \ 功能项 | 结构迁移 | 全量数据迁移 | 增量实时同步 | 数据校验 |
+ | --- | --- | --- | --- | --- |
+ | Oracle 源端 | 支持 | 支持 | 支持 | 支持 |
+ | PostgreSQL 源端 | 支持 | 支持 | 支持 | 支持 |
+ | Greenplum 源端 | 支持 | 支持 | 不支持 | 支持 |
+ | MySQL 源端 | 支持 | 支持 | 支持 | 支持 |
+
+## 使用方法
+
+CloudCanal 提供了完整的产品化能力，用户在可视化界面完成数据源添加和任务创建即可自动完成结构迁移、全量迁移、增量实时同步。下文演示如何将 MySQL 数据库中的数据迁移同步到对端 Doris 中。其他源端同步到 Doris 也可以按照类似的方式进行。
+
+### 前置条件
+
+首先参考 [CloudCanal 安装部署参考文档](https://www.askcug.com/topic/75) 完成 CloudCanal 社区版的安装和部署。
+
+### 添加数据源
+
+- 登录 CloudCanal 平台
+- 数据源管理-> 新增数据源
+- 选择自建数据库中 Doris
+
+
+
+> Tips:
+>
+> - Client 地址： 为 Doris 提供给 MySQL Client 的服务端口，CloudCanal 主要用其查询库表的元数据信息
+>
+> - Http 地址： Http 地址主要用于接收 CloudCanal 数据导入的请求
+
+### 任务创建
+
+添加好数据源之后可以按照如下步骤进行数据迁移、同步任务的创建。
+
+- **任务管理**-> **任务创建**
+- 选择 **源** 和 **目标** 数据库
+- 点击 下一步
+
+
+
+- 选择 **增量同步**，并且启用 **全量数据初始化**
+- 不勾选 DDL 同步(暂不支持)
+- 点击下一步
+
+
+
+- 选择订阅的表，**结构迁移自动创建的表为主键模型的表，因此暂不支持无主键表**
+- 点击下一步
+
+
+
+- 配置列映射
+- 点击下一步
+
+
+
+- 创建任务
+
+
+
+- 查看任务状态。任务创建后，会自动完成结构迁移、全量、增量阶段。
+
+
+
+## 参考资料
+
+更多关于 CloudCanal 同步 Doris 的资料，可以查看
+
+- [5 分钟搞定 MySQL/PostgreSQL/Oracle 到 Doris 数据迁移同步-CloudCanal 实战](https://www.askcug.com/topic/262)
+
+- [CloudCanal 官方社区](https://www.askcug.com/)
diff --git a/sidebars.json b/sidebars.json
index a568ca45eb..c224f245e6 100644
--- a/sidebars.json
+++ b/sidebars.json
@@ -187,6 +187,7 @@
"ecosystem/spark-doris-connector",
"ecosystem/datax",
"ecosystem/logstash",
+ "ecosystem/cloudcanal",
{
"type": "category",
"label": "Doris Manager",
diff --git a/static/images/cloudcanal/cloudcanal-1.jpg b/static/images/cloudcanal/cloudcanal-1.jpg
new file mode 100644
index 0000000000..978fdd95ad
Binary files /dev/null and b/static/images/cloudcanal/cloudcanal-1.jpg differ
diff --git a/static/images/cloudcanal/cloudcanal-2.jpg b/static/images/cloudcanal/cloudcanal-2.jpg
new file mode 100644
index 0000000000..2e58d25ca5
Binary files /dev/null and b/static/images/cloudcanal/cloudcanal-2.jpg differ
diff --git a/static/images/cloudcanal/cloudcanal-3.jpg b/static/images/cloudcanal/cloudcanal-3.jpg
new file mode 100644
index 0000000000..e3050e59ed
Binary files /dev/null and b/static/images/cloudcanal/cloudcanal-3.jpg differ
diff --git a/static/images/cloudcanal/cloudcanal-4.png b/static/images/cloudcanal/cloudcanal-4.png
new file mode 100644
index 0000000000..379c95018e
Binary files /dev/null and b/static/images/cloudcanal/cloudcanal-4.png differ
diff --git a/static/images/cloudcanal/cloudcanal-5.png b/static/images/cloudcanal/cloudcanal-5.png
new file mode 100644
index 0000000000..ec9518bf95
Binary files /dev/null and b/static/images/cloudcanal/cloudcanal-5.png differ
diff --git a/static/images/cloudcanal/cloudcanal-6.png b/static/images/cloudcanal/cloudcanal-6.png
new file mode 100644
index 0000000000..c1e7be356c
Binary files /dev/null and b/static/images/cloudcanal/cloudcanal-6.png differ
diff --git a/static/images/cloudcanal/cloudcanal-7.png b/static/images/cloudcanal/cloudcanal-7.png
new file mode 100644
index 0000000000..62ae33a555
Binary files /dev/null and b/static/images/cloudcanal/cloudcanal-7.png differ
diff --git a/static/images/cloudcanal/cloudcanal-8.jpg b/static/images/cloudcanal/cloudcanal-8.jpg
new file mode 100644
index 0000000000..aa4cae7c5e
Binary files /dev/null and b/static/images/cloudcanal/cloudcanal-8.jpg differ
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org