You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by ki...@apache.org on 2021/12/13 03:51:25 UTC

[dolphinscheduler-website] branch master updated: add datasource spi design doc (#6071) (#500)

This is an automated email from the ASF dual-hosted git repository.

kirs pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git


The following commit(s) were added to refs/heads/master by this push:
     new 457ae9f  add datasource spi design doc (#6071) (#500)
457ae9f is described below

commit 457ae9f48506fabd661ad3a752d13b49ccdfd6e3
Author: mask <39...@users.noreply.github.com>
AuthorDate: Mon Dec 13 11:51:20 2021 +0800

    add datasource spi design doc (#6071) (#500)
    
    * add datasource spi design doc
    
    * add datasource spi design doc
    
    * add datasource spi design doc
---
 development/en-us/backend/spi/datasource.md | 23 +++++++++++++++++++++++
 development/zh-cn/backend/spi/datasource.md | 23 +++++++++++++++++++++++
 2 files changed, 46 insertions(+)

diff --git a/development/en-us/backend/spi/datasource.md b/development/en-us/backend/spi/datasource.md
new file mode 100644
index 0000000..5772b43
--- /dev/null
+++ b/development/en-us/backend/spi/datasource.md
@@ -0,0 +1,23 @@
+## DolphinScheduler Datasource SPI main design
+
+#### How do I use data sources?
+
+The data source center supports POSTGRESQL, HIVE/IMPALA, SPARK, CLICKHOUSE, SQLSERVER data sources by default.
+
+If you are using MySQL or ORACLE data source, you need to place the corresponding driver package in the lib directory
+
+#### How to do Datasource plugin development?
+
+org.apache.dolphinscheduler.spi.datasource.DataSourceChannel
+org.apache.dolphinscheduler.spi.datasource.DataSourceChannelFactory
+org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient
+
+1. In the first step, the data source plug-in can implement the above interfaces and inherit the general client. For details, refer to the implementation of data source plug-ins such as sqlserver and mysql. The addition methods of all RDBMS plug-ins are the same.
+
+2. Add the driver configuration in the data source plug-in pom.xml
+
+We provide APIs for external access of all data sources in the dolphin scheduler data source API module
+
+#### **Future plan**
+
+Support data sources such as kafka, http, files, sparkSQL, FlinkSQL, etc.
\ No newline at end of file
diff --git a/development/zh-cn/backend/spi/datasource.md b/development/zh-cn/backend/spi/datasource.md
new file mode 100644
index 0000000..1868c86
--- /dev/null
+++ b/development/zh-cn/backend/spi/datasource.md
@@ -0,0 +1,23 @@
+## DolphinScheduler Datasource SPI 主要设计
+
+#### 如何使用数据源?
+
+数据源中心默认支持POSTGRESQL、HIVE/IMPALA、SPARK、CLICKHOUSE、SQLSERVER数据源。
+
+如果使用的是MySQL、ORACLE数据源则需要、把对应的驱动包放置lib目录下
+
+#### 如何进行数据源插件开发?
+
+org.apache.dolphinscheduler.spi.datasource.DataSourceChannel
+org.apache.dolphinscheduler.spi.datasource.DataSourceChannelFactory
+org.apache.dolphinscheduler.plugin.datasource.api.client.CommonDataSourceClient
+
+1. 第一步数据源插件实现以上接口和继承通用client即可,具体可以参考sqlserver、mysql等数据源插件实现,所有RDBMS插件的添加方式都是一样的。
+2. 在数据源插件pom.xml添加驱动配置
+
+我们在 dolphinscheduler-datasource-api 模块提供了所有数据源对外访问的 API
+
+#### **未来计划**
+
+支持kafka、http、文件、sparkSQL、FlinkSQL等数据源
+