You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by zh...@apache.org on 2021/12/14 11:04:16 UTC
[dolphinscheduler-website] branch master updated: Update plugin doc (#566)
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler-website.git
The following commit(s) were added to refs/heads/master by this push:
new 990e856 Update plugin doc (#566)
990e856 is described below
commit 990e856e4e1a3cb918fdb4297487283be8639973
Author: Kirs <ac...@163.com>
AuthorDate: Tue Dec 14 19:03:29 2021 +0800
Update plugin doc (#566)
---
development/en-us/backend/spi/alert.md | 75 ++++++----------------
development/en-us/backend/spi/plugin.md | 54 ----------------
development/en-us/backend/spi/registry.md | 35 +++++-----
development/en-us/backend/spi/task.md | 35 ++--------
development/en-us/development-environment-setup.md | 44 +++++--------
development/zh-cn/backend/spi/alert.md | 48 ++------------
development/zh-cn/backend/spi/plugin.md | 54 ----------------
development/zh-cn/backend/spi/registry.md | 32 ++++-----
development/zh-cn/backend/spi/task.md | 28 ++------
development/zh-cn/development-environment-setup.md | 46 +++++--------
10 files changed, 98 insertions(+), 353 deletions(-)
diff --git a/development/en-us/backend/spi/alert.md b/development/en-us/backend/spi/alert.md
index b149afb..d5e94bc 100644
--- a/development/en-us/backend/spi/alert.md
+++ b/development/en-us/backend/spi/alert.md
@@ -2,71 +2,53 @@
#### DolphinScheduler SPI Design
-The architecture of microkernel & plug-in of DolphinScheduler is changing. All core capabilities such as tasks, resource storage, registry, etc. will be designed to be extensions, and we want to improve the flexibility as well as the friendliness (extensibility) of DolphinScheduler itself through SPI.
+DolphinScheduler is undergoing a microkernel + plug-in architecture change. All core capabilities such as tasks, resource storage, registration centers, etc. will be designed as extension points. We hope to use SPI to improve DolphinScheduler’s own flexibility and friendliness (extended sex).
-You can read the relevant code under the dolphinscheduler-spi module as a reference. The extended interface of the associated plug-in is under the module, and when we need to implement the relevant function plug-in, it is recommended to read this code block first. Of course, you are welcomed to read the documentation, which will save a lot less time. However, the documentation has a certain lag and when it is missing, it is suggested to take the source code as the reference (If you are i [...]
+For alarm-related codes, please refer to the `dolphinscheduler-alert-api` module. This module defines the extension interface of the alarm plug-in and some basic codes. When we need to realize the plug-inization of related functions, it is recommended to read the code of this block first. Of course, it is recommended that you read the document. This will reduce a lot of time, but the document There is a certain degree of lag. When the document is missing, it is recommended to take the so [...]
-When you need to extend, you actually only need to focus on the extension interface. For example, for the alert service you only need to focus on the AlertChannelFactory and the AlertChannel. The underlying logic is already implemented by DolphinScheduler, which makes our development much more focused and simpler.
+We use the native JAVA-SPI, when you need to extend, in fact, you only need to pay attention to the extension of the `org.apache.dolphinscheduler.alert.api.AlertChannelFactory` interface, the underlying logic such as plug-in loading, and other kernels have been implemented, Which makes our development more focused and simple.
-By the way, we use an excellent front-end component, form-create, which supports the generation of front-end ui components based on json. If plugin development involves a front-end, we will use json to generate the relevant front-end UI components. The parameters of the plugin are wrapped in org.apache.dolphinscheduler.spi.params, which converts all the relevant parameters into json, meaning that you can draw the front-end components (mainly forms here, we only care about the data of fro [...]
+By the way, we have adopted an excellent front-end component form-create, which supports the generation of front-end UI components based on JSON. If plug-in development involves the front-end, we will use JSON to generate related front-end UI components, org.apache.dolphinscheduler. The parameters of the plug-in are encapsulated in spi.params, which will convert all the relevant parameters into the corresponding JSON, which means that you can complete the drawing of the front-end compone [...]
-This article focuses on the design and development of Alert alerts.
+This article mainly focuses on the design and development of Alert.
#### Main Modules
-If you don't care about its internal design, and just want to know how to develop your own alarm plugin, you can skip this section.
+If you don't care about its internal design, but simply want to know how to develop your own alarm plug-in, you can skip this content.
-* dolphinscheduler-spi
+* dolphinscheduler-alert-api
- This module is the core SPI module and provides the basic SPI relevant behaviour, where dolphinschedulerplugin is the plugin top-layer interface, all DolphinsCheduler's plugins must implement the interface, and the module also provides some universal tool classes (if you can detach it again Some will something will be better? For example, the UI, we currently use the parameter part) and some UI related basic information.
+ This module is the core module of ALERT SPI. This module defines the interface of the alarm plug-in extension and some basic codes. The extension plug-in must implement the interface defined by this module: `org.apache.dolphinscheduler.alert.api.AlertChannelFactory`
-* dolphinscheduler-alert
-
- In this module, we have implemented the load of the associated plugin when the Alert-Server starts. Alert provides a variety of plug-in configuration methods that can be enabled by simple configurations after you have done the development. The configuration file is located at dolphinscheduler-alert/src/main/resources/alert.properties .
-
- It provides two methods of configuration.
-
- 1: Configure the jar directory specified by the plugin, eg: alert.plugin.dir=/root/dolphinscheduler/lib/plugin/alert . When alert-server starts, it will load the jar of the relevant plugin from the specified directory.
-
- 2: IDE development mode
-
- You can use this configuration when you are in the debugging phase of development, see [dolphinscheduler-maven-plugin](https://github.com/apache/incubator-dolphinscheduler-maven-plugin) for design principles.
-
-
-
- * Packaging plugins
-
- We use provisio, an excellent packaging tool, for packaging plugins. Please add it to dolphinscheduler-dist/src/main/provisio/dolphinscheduler.xml, and it will package plugins to the specified directory when processed.
+* dolphinscheduler-alert-plugins
+ This module is currently a plug-in provided by us, such as Email, DingTalk, Script, etc.
#### Alert SPI Main class information.
-
AlertChannelFactory
-All alert plugins need to implement this interface. The interface is used to define the name of the alert plugin and the required parameters.
+Alarm plug-in factory interface. All alarm plug-ins need to implement this interface. This interface is used to define the name of the alarm plug-in and the required parameters. The create method is used to create a specific alarm plug-in instance.
AlertChannel
-The interface of the alarm plug-in. The alarm plugin needs to implement the interface. It only has one method process. The upper alarm system calls the method and gets the return information of the alert by the AlertResult returned by this method.
+The interface of the alert plug-in. The alert plug-in needs to implement this interface. There is only one method process in this interface. The upper-level alert system will call this method and obtain the return information of the alert through the AlertResult returned by this method.
AlertData
-Alert content information. It includes id, title, content and log.
+Alarm content information, including id, title, content, log.
AlertInfo
-Alarm-related information. When the upper system calls the alarm plug-in instance, the instance of this class is incorporated into the specific alarm plugin through the Process method. It contains the parameter information filled in the front end of the alarm content AlertData and the called alarm plugin instance.
+For alarm-related information, when the upper-level system calls an instance of the alarm plug-in, the instance of this class is passed to the specific alarm plug-in through the process method. It contains the alert content AlertData and the parameter information filled in by the front end of the called alert plug-in instance.
AlertResult
-The alert plugin sends an alert return message.
+The alarm plug-in sends alarm return information.
-org.apache.dolphinscheduler.spi.params
-This package contains the plug-in parameter definitions. We use alpacajs, a front-end library http://www.form-create.com, which dynamically generates the front-end ui based on the parameter list json returned by the plug-in definition, so we don't need to care about the front-end when doing SPI plug-in development.
+org.apache.dolphinscheduler.spi.params
+This package is a plug-in parameter definition. Our front-end uses the from-create front-end library http://www.form-create.com, which can dynamically generate the front-end UI based on the parameter list json returned by the plug-in definition, so We don't need to care about the front end when we are doing SPI plug-in development.
-Inside this package we currently only wrap RadioParam, TextParam and PasswordParam, which are used to define parameters of text, radio and password respectively.
+Under this package, we currently only encapsulate RadioParam, TextParam, and PasswordParam, which are used to define text type parameters, radio parameters and password type parameters, respectively.
-AbsPluginParams
+AbsPluginParams This class is the base class of all parameters, RadioParam these classes all inherit this class. Each DS alert plug-in will return a list of AbsPluginParams in the implementation of AlertChannelFactory.
-This class is the base class for all parameters and is inherited by the RadioParam classes. Each DS alert plugin returns a list of AbsPluginParams in the AlertChannelFactory implementation.
-
-The specific design of alert_spi can be found in issue: [Alert Plugin Design](https://github.com/apache/incubator-dolphinscheduler/issues/3049)
+The specific design of alert_spi can be seen in the issue: [Alert Plugin Design](https://github.com/apache/incubator-dolphinscheduler/issues/3049)
#### Alert SPI built-in implementation
@@ -91,20 +73,3 @@ The specific design of alert_spi can be found in issue: [Alert Plugin Design](ht
* SMS
SMS alerts
-
-#### Alarm Custom Plugin Development
-
-In fact, it's very easy to implement a plugin by yourself, you only need to care about the plugin extension interface. In Alert you only need to care about the AlertChannelFactory and AlertChannel. We would recommend that you follow the design specifications of other built-in plugins so that when your idea is good enough, you can donate it to the community without having to change it too much.
-
-When you complete the development of the relevant code, you need to execute `mvn -U install -Dmaven.test.skip=true` to install the plug-in and generate the plug-in jar of the alert. The directory is: dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/alert (the version number will change with the main version number)
-
-Note: **${VERSION}** needs to be manually modified according to the current version. regarding alert.plugin.binding, maven.local.repository does not need to be modified.
-
-alert.properties configuration
-```
-alert.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/alert
-```
-
-Then, you can happily start using your own plugins.
-
-In fact, custom plug-in development is as easy as we described, and not as difficult as you imagined.
diff --git a/development/en-us/backend/spi/plugin.md b/development/en-us/backend/spi/plugin.md
deleted file mode 100644
index 46050a9..0000000
--- a/development/en-us/backend/spi/plugin.md
+++ /dev/null
@@ -1,54 +0,0 @@
-## Task Plugin Development
-
-Remind:Currently, task plugin development does not support hot deployment.
-
-### Shell-based tasks
-
-#### YARN-based calculations (see MapReduceTask)
-
-- Need to be **org.apache.dolphinscheduler.server.worker.task** Down **TaskManager** Create a custom task in the class (also need to register the corresponding task type in TaskType)
-- Need to inherit**org.apache.dolphinscheduler.server.worker.task** Down **AbstractYarnTask**
-- Constructor Scheduling **AbstractYarnTask** Construction method
-- Inherit **AbstractParameters** Custom task parameter entity
-- Rewrite **AbstractTask** of **init** Parsing in method**Custom task parameters**
-- Rewrite **buildCommand** Encapsulation command
-
-
-
-#### Non-YARN-based calculations (see ShellTask)
-- Need to be **org.apache.dolphinscheduler.server.worker.task** Down **TaskManager** A custom task
-
-- Need to inherit**org.apache.dolphinscheduler.server.worker.task** Down **AbstractTask**
-
-- Instantiation in constructor **ShellCommandExecutor**
-
- ```
- public ShellTask(TaskProps props, Logger logger) {
- super(props, logger);
-
- this.taskDir = props.getTaskDir();
-
- this.processTask = new ShellCommandExecutor(this::logHandle,
- props.getTaskDir(), props.getTaskAppId(),
- props.getTenantCode(), props.getEnvFile(), props.getTaskStartTime(),
- props.getTaskTimeout(), logger);
- this.processDao = DaoFactory.getDaoInstance(ProcessDao.class);
- }
- ```
-
- Incoming custom tasks **TaskProps**And custom**Logger**，TaskProps Encapsulate task information, Logger is installed with custom log information
-
-- Inherit **AbstractParameters** Custom task parameter entity
-
-- Rewrite **AbstractTask** of **init** Parsing in method**Custom task parameter entity**
-
-- Rewrite **handle** method，transfer **ShellCommandExecutor** of **run** method，The first parameter is passed in**command**，Pass the second parameter to ProcessDao and set the corresponding **exitStatusCode**
-
-### Non-SHELL-based tasks (see SqlTask)
-
-- Need to be **org.apache.dolphinscheduler.server.worker.task** Down **TaskManager** A custom task
-- Need to inherit**org.apache.dolphinscheduler.server.worker.task** Down **AbstractTask**
-- Inherit **AbstractParameters** Custom task parameter entity
-- Constructor or override **AbstractTask** of **init** in the method, parse the custom task parameter entity
-- Rewrite **handle** Methods to implement business logic and set the corresponding**exitStatusCode**
-
diff --git a/development/en-us/backend/spi/registry.md b/development/en-us/backend/spi/registry.md
index aaf1d06..0957ff3 100644
--- a/development/en-us/backend/spi/registry.md
+++ b/development/en-us/backend/spi/registry.md
@@ -1,28 +1,27 @@
-### DolphinScheduler Registry SPI main design
+### DolphinScheduler Registry SPI Extension
-#### How to use it？
+#### how to use?
-Note: **${VERSION}** needs to be manually modified according to the current version.
+Make the following configuration (take zookeeper as an example)
-First you need to execute the `mvn -U install -Prelease -Dmaven.test.skip=true` to install the plugin for generating the plugin JAR of the registry. The directory is: dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/registry.
+* Registry plug-in configuration, take Zookeeper as an example (registry.properties)
+ dolphinscheduler-service/src/main/resources/registry.properties
+ ```registry.properties
+ registry.plugin.name=zookeeper
+ registry.servers=127.0.0.1:2181
+ ```
-Next, please follow the configuration below (using zookeeper as an example).
+For specific configuration information, please refer to the parameter information provided by the specific plug-in, for example zk: `org/apache/dolphinscheduler/plugin/registry/zookeeper/ZookeeperConfiguration.java`
+All configuration information prefixes need to be +registry, such as base.sleep.time.ms, which should be configured in the registry as follows: registry.base.sleep.time.ms=100
-| parameter | default | description |
-| :---------------------: | :----------------------------------------------------------: | :--------------------------------------: |
-| registry.plugin.dir | ./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/registry | Registration Center Plugin Directory |
-| registry.plugin.name | zookeeper | Registration Center specific plugin name |
-| registry.plugin.binding | registry | Dolphinscheduler plugin category |
-| registry.servers | 127.0.0.1:2181 | ZK connection address |
+#### How to expand
-For the specific configuration information, please refer to the parameter information that provided by the specific plugin. Taking zk as an example, all the parameters of this configuration information about zk are in the class org/apache/dolphinscheduler/plugin/registry/zookeeper/ZookeeperConfiguration.java. If a parameter is needed to be changed to a specific value instead of the default, it can be configured directly in the registry. However, the prefix should be added to indicate thi [...]
+`dolphinscheduler-registry-api` defines the standard for implementing plugins. When you need to extend plugins, you only need to implement `org.apache.dolphinscheduler.registry.api.RegistryFactory`.
-#### FAQ
-
-1: registry plugin not found
+Under the `dolphinscheduler-registry-plugin` module is the registry plugin we currently provide.
-Please check if `mvn -U install -Dmaven.test.skip=true` is executed. Besides, please check if the directory is configured in registry.plugin.dir in the configuration file contains the relevant plugins.
+#### FAQ
-2：registry connect timeout
+1: registry connect timeout
-You can add the relevant timeout parameters.
+You can increase the relevant timeout parameters.
diff --git a/development/en-us/backend/spi/task.md b/development/en-us/backend/spi/task.md
index 467bdda..52650db 100644
--- a/development/en-us/backend/spi/task.md
+++ b/development/en-us/backend/spi/task.md
@@ -1,36 +1,9 @@
-## DolphinScheduler Task SPI main design
+## DolphinScheduler Task SPI extension
-#### How to use development environment?
-
-Note: **${VERSION}** needs to be manually modified according to the current version.
-
-First you need to execute the `mvn -U install -Dmaven.test.skip=true` to install the plugin for generating the plugin JAR of the registry. The directory is: dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/task (The version number will follow the change of the main version number)
-
-Executing this command will generate all plugins provided by default for all modules.
-
-Next, configure the plugin directory in (dolphinscheduler-server/src/main/resources/worker.properties)
-
-```
-task.plugin.dir config the #task.plugin.dir config the Task Plugin dir . WorkerServer while find and load the Task Plugin Jar from this dir when deploy and start WorkerServer on the server .
-task.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/task
-```
-
-The following configurations can be used for local development debugging, for example:
-
-```
-maven.local.repository=/Users/localRepository
-#task.plugin.binding config the task plugin need be load when development and run in IDE
-task.plugin.binding=./dolphinscheduler-task-plugin/dolphinscheduler-task-shell/pom.xml
-```
-
-Please configure according to the actual situation.
-
-#### How to do task plugin development?
-
-### Back-end development:
+#### How to develop task plugins?
org.apache.dolphinscheduler.spi.task.TaskChannel
-The plugin should implements the above interface. It mainly contains methods for creating tasks (task initialization, task run, etc.), task cancellation, and if it is a yarn task, it needs to implement org.apache.dolphinscheduler.plugin.task.api.AbstractYarnTask.
+The plug-in can implement the above interface. It mainly includes creating tasks (task initialization, task running, etc.) and task cancellation. If it is a yarn task, you need to implement org.apache.dolphinscheduler.plugin.task.api.AbstractYarnTask.
-We provide the API for all tasks to be accessed externally in the dolphinscheduler-task-api module. And dolphinscheduler-spi module is the spi common code base that defines all the plug-in modules like alert module, registry module etc. You can read it in detail to see it.
+We provide APIs for external access to all tasks in the dolphinscheduler-task-api module, while the dolphinscheduler-spi module is the spi general code library, which defines all the plug-in modules, such as the alarm module, the registry module, etc., you can read and view in detail .
diff --git a/development/en-us/development-environment-setup.md b/development/en-us/development-environment-setup.md
index 3d00801..30973f3 100644
--- a/development/en-us/development-environment-setup.md
+++ b/development/en-us/development-environment-setup.md
@@ -30,7 +30,6 @@ There are two ways to configure the DolphinScheduler development environment, st
## DolphinScheduler Standalone Quick Start
> **_Note:_** Standalone server only for development and debugging, cause it use H2 Database, Zookeeper Testing Server which may not stable in production
-> If you want to test plugin, you can modify `plugin.bind` in StandaloneServer class or modify the configuration file by yourself.
> Standalone is only supported in DolphinScheduler 1.3.9 and later versions
### Git Branch Choose
@@ -42,12 +41,6 @@ Use different Git branch to develop different codes
### Start backend server
-Compile backend code
-
-```shell
-mvn install -DskipTests
-```
-
Find the class `org.apache.dolphinscheduler.server.StandaloneServer` in Intellij IDEA and clikc run main function to startup.
### Start frontend server
@@ -94,32 +87,27 @@ Following steps will guide how to start the DolphinScheduler backend service
##### Backend Start Prepare
* Open project: Use IDE open the project, here we use Intellij IDEA as an example, after opening it will take a while for Intellij IDEA to complete the dependent download
-* Plugin installation(**Only required for 2.0 or later**): Compile plugin by command `mvn -U clean install -Dmaven.test.skip=true`
-
- Note: **${VERSION}** needs to be manually modified according to the current version, regarding ***.plugin.binding, maven.local.repository does not need to be modified.
+* Plugin installation(**Only required for 2.0 or later**)
- * alert plugin config (alert.properties)
- ```alert.properties
- alert.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/alert
- ```
- * registry plugin config (registry.properties)
+ * Registry plug-in configuration, take Zookeeper as an example (registry.properties)
+ dolphinscheduler-service/src/main/resources/registry.properties
```registry.properties
- registry.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/registry/zookeeper
- ```
- * task plugin config (worker.properties)
- ```worker.properties
- task.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/task
+ registry.plugin.name=zookeeper
+ registry.servers=127.0.0.1:2181
```
* File change
* If you use MySQL as your metadata database, you need to modify `dolphinscheduler/pom.xml` and change the `scope` of the `mysql-connector-java` dependency to `compile`. This step is not necessary to use PostgreSQL
- * Modify database configuration, modify the database configuration in the `dolphinscheduler/dolphinscheduler-dao/datasource.properties`
-
- ```properties
- # We here use MySQL with database, username, password named dolphinscheduler as an example
- spring.datasource.driver-class-name=com.mysql.jdbc.Driver
- spring.datasource.url=jdbc:mysql://localhost:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true
- spring.datasource.username=dolphinscheduler
- spring.datasource.password=dolphinscheduler
+ * Modify database configuration, modify the database configuration in the `dolphinscheduler-dao/src/main/resources/application-mysql.yaml`
+
+
+ We here use MySQL with database, username, password named dolphinscheduler as an example
+ ```application-mysql.yaml
+ spring:
+ datasource:
+ driver-class-name: com.mysql.jdbc.Driver
+ url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
+ username: ds_user
+ password: dolphinscheduler
```
* Log level: add a line `<appender-ref ref="STDOUT"/>` to the following configuration to enable the log to be displayed on the command line
diff --git a/development/zh-cn/backend/spi/alert.md b/development/zh-cn/backend/spi/alert.md
index 6015288..04890d0 100644
--- a/development/zh-cn/backend/spi/alert.md
+++ b/development/zh-cn/backend/spi/alert.md
@@ -4,9 +4,9 @@
DolphinScheduler 正在处于微内核 + 插件化的架构更改之中，所有核心能力如任务、资源存储、注册中心等都将被设计为扩展点，我们希望通过 SPI 来提高 DolphinScheduler 本身的灵活性以及友好性（扩展性）。
-相关代码可以参考 dolphinscheduler-spi 模块。相关插件的扩展接口皆在该模块下，当我们需要实现相关功能的插件化的时候，建议先阅读此块的代码，当然，更建议你阅读文档，这会减少很多时间，不过文档有一定的后滞性，当文档缺失的时候，建议以源码为准（如果有兴趣，我们也欢迎你来提交相关文档），此外，我们几乎不会对扩展接口做变更（不包括新增），除非重大架构调整，出现不兼容升级版本，因此，现有文档一般都能够满足。
+告警相关代码可以参考 `dolphinscheduler-alert-api` 模块。该模块定义了告警插件扩展的接口以及一些基础代码，当我们需要实现相关功能的插件化的时候，建议先阅读此块的代码，当然，更建议你阅读文档，这会减少很多时间，不过文档有一定的后滞性，当文档缺失的时候，建议以源码为准（如果有兴趣，我们也欢迎你来提交相关文档），此外，我们几乎不会对扩展接口做变更（不包括新增），除非重大架构调整，出现不兼容升级版本，因此，现有文档一般都能够满足。
-当你需要扩展的时候，事实上你只需要关注扩展接口即可，比如告警服务，您只需要关注 AlertChannelFactory 以及 AlertChannel 即可。底层相关逻辑 DolphinScheduler 已经帮我们实现，这让我们的开发更加专注且简单。
+我们采用了原生的 JAVA-SPI，当你需要扩展的时候，事实上你只需要关注扩展`org.apache.dolphinscheduler.alert.api.AlertChannelFactory`接口即可，底层相关逻辑如插件加载等内核已经实现，这让我们的开发更加专注且简单。
顺便提一句，我们采用了一款优秀的前端组件 form-create，它支持基于 json 生成前端 ui 组件，如果插件开发牵扯到前端，我们会通过 json 来生成相关前端 UI 组件，org.apache.dolphinscheduler.spi.params 里面对插件的参数做了封装，它会将相关参数全部全部转化为对应的 json，这意味这你完全可以通过 Java 代码的方式完成前端组件的绘制（这里主要是表单，我们只关心前后端交互的数据）。
@@ -16,27 +16,13 @@ DolphinScheduler 正在处于微内核 + 插件化的架构更改之中，所有
如果你并不关心它的内部设计，只是想单纯的了解如何开发自己的告警插件，可以略过该内容。
-* dolphinscheduler-spi
+* dolphinscheduler-alert-api
- 该模块是 SPI 的核心模块，提供了 SPI 相关的基础行为，其中 DolphinSchedulerPlugin 为插件顶层接口，所有 DolphinScheduler 的插件都必须实现该接口，另外该模块也提供了一些通用的工具类（如果能够再抽离出去一些会不会更好？比如 UI，我们目前就 Alert 用到了参数这块）以及一些 UI 相关的基础信息。
+ 该模块是 ALERT SPI 的核心模块，该模块定义了告警插件扩展的接口以及一些基础代码，扩展插件必须实现此模块所定义的接口:`org.apache.dolphinscheduler.alert.api.AlertChannelFactory`
-* dolphinscheduler-alert
-
- 在这个模块，我们实现了在 alert-server 启动的时候相关插件的加载。alert 提供了多种插件配置方法，当你开发工作完成后，通过简单的配置即可启用。配置文件位于 dolphinscheduler-alert/src/main/resources/alert.properties
-
- 它提供了两种配置方法：
-
- 1：配置插件指定的 jar 目录，eg：alert.plugin.dir=/root/dolphinscheduler/lib/plugin/alert . 当alert-server启动时，会从指定目录加载相关插件的jar。
-
- 2：IDE开发模式
-
- 当你处于开发调试阶段的时候，你可以采用该配置，相关设计原理参照 [dolphinscheduler-maven-plugin](https://github.com/apache/incubator-dolphinscheduler-maven-plugin)
-
-
- * 打包插件
-
- 插件打包我们使用了 [provisio](https://github.com/jvanzyl/provisio)，这是一款优秀的打包工具，你需要在完成插件开发后将其添加到 dolphinscheduler-dist/src/main/provisio/dolphinscheduler.xml，它会在执行的时候将插件打包至指定目录。
+* dolphinscheduler-alert-plugins
+ 该模块是目前我们提供的插件，如 Email、DingTalk、Script等。
#### Alert SPI 主要类信息：
@@ -57,7 +43,7 @@ AlertResult
告警插件发送告警返回信息。
org.apache.dolphinscheduler.spi.params
-该包下是插件化的参数定义，我们前端使用 alpacajs 这个前端库 http://www.form-create.com，该库可以基于插件定义返回的参数列表 json 来动态生成前端的 ui，因此我们在做 SPI 插件开发的时候无需关心前端。
+该包下是插件化的参数定义，我们前端使用 from-create 这个前端库 http://www.form-create.com，该库可以基于插件定义返回的参数列表 json 来动态生成前端的 ui，因此我们在做 SPI 插件开发的时候无需关心前端。
该 package 下我们目前只封装了 RadioParam，TextParam，PasswordParam，分别用来定义 text 类型的参数，radio 参数和 password 类型的参数。
@@ -83,23 +69,3 @@ alert_spi 具体设计可见 issue：[Alert Plugin Design](https://github.com/ap
* Script
我们实现了 Shell 脚本告警，我们会将相关告警参数透传给脚本，你可以在 Shell 中实现你的相关告警逻辑，如果你需要对接内部告警应用，这是一种不错的方法。
-
-* SMS
-
- 短信告警
-
-#### 告警自定义插件开发
-
-事实上，自我实现一款插件及其简单，仅仅关心插件扩展接口即可，Alert 中你只需要关心 AlertChannelFactory 以及 AlertChannel。我们更建议你按照其他内置插件的设计规范来去开发，这样当你的idea足够好的时候，你无需做过多更改即可捐献给社区。
-
-注意：**${VERSION}** 需要根据当前版本手动修改。
-
-当你完成相关代码开发的时候, 你需要执行 `mvn -U install -Dmaven.test.skip=true` 安装插件，生成注册中心的插件 jar。目录是：dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/alert（版本号会跟随主版本号变更）
-
-alert.properties 配置，对于 alert.plugin.binding,maven.local.repository 这两个参数则无需做修改。
-```
-alert.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/alert
-```
-然后，接下来就可以开始愉快的使用你自己的插件了。
-
-事实上，自定义插件化开发确实如同我们描绘的那么简单，并没有想象中的多么困难。
diff --git a/development/zh-cn/backend/spi/plugin.md b/development/zh-cn/backend/spi/plugin.md
deleted file mode 100644
index 5e899aa..0000000
--- a/development/zh-cn/backend/spi/plugin.md
+++ /dev/null
@@ -1,54 +0,0 @@
-## 任务插件开发
-
-提醒:目前任务插件开发暂不支持热部署
-
-### 基于SHELL的任务
-
-#### 基于YARN的计算（参见MapReduceTask）
-
-- 需要在 **org.apache.dolphinscheduler.server.worker.task** 下的 **TaskManager** 类中创建自定义任务(也需在TaskType注册对应的任务类型)
-- 需要继承**org.apache.dolphinscheduler.server.worker.task** 下的 **AbstractYarnTask**
-- 构造方法调度 **AbstractYarnTask** 构造方法
-- 继承 **AbstractParameters** 自定义任务参数实体
-- 重写 **AbstractTask** 的 **init** 方法中解析**自定义任务参数**
-- 重写 **buildCommand** 封装command
-
-
-
-#### 基于非YARN的计算（参见ShellTask）
-- 需要在 **org.apache.dolphinscheduler.server.worker.task** 下的 **TaskManager** 中创建自定义任务
-
-- 需要继承**org.apache.dolphinscheduler.server.worker.task** 下的 **AbstractTask**
-
-- 构造方法中实例化 **ShellCommandExecutor**
-
- ```
- public ShellTask(TaskProps props, Logger logger) {
- super(props, logger);
-
- this.taskDir = props.getTaskDir();
-
- this.processTask = new ShellCommandExecutor(this::logHandle,
- props.getTaskDir(), props.getTaskAppId(),
- props.getTenantCode(), props.getEnvFile(), props.getTaskStartTime(),
- props.getTaskTimeout(), logger);
- this.processDao = DaoFactory.getDaoInstance(ProcessDao.class);
- }
- ```
-
- 传入自定义任务的 **TaskProps**和自定义**Logger**，TaskProps 封装了任务的信息，Logger封装了自定义日志信息
-
-- 继承 **AbstractParameters** 自定义任务参数实体
-
-- 重写 **AbstractTask** 的 **init** 方法中解析**自定义任务参数实体**
-
-- 重写 **handle** 方法，调用 **ShellCommandExecutor** 的 **run** 方法，第一个参数传入自己的**command**，第二个参数传入 ProcessDao，设置相应的 **exitStatusCode**
-
-### 基于非SHELL的任务（参见SqlTask）
-
-- 需要在 **org.apache.dolphinscheduler.server.worker.task** 下的 **TaskManager** 中创建自定义任务
-- 需要继承**org.apache.dolphinscheduler.server.worker.task** 下的 **AbstractTask**
-- 继承 **AbstractParameters** 自定义任务参数实体
-- 构造方法或者重写 **AbstractTask** 的 **init** 方法中，解析自定义任务参数实体
-- 重写 **handle** 方法实现业务逻辑并设置相应的**exitStatusCode**
-
diff --git a/development/zh-cn/backend/spi/registry.md b/development/zh-cn/backend/spi/registry.md
index 7c067c2..36c4d1f 100644
--- a/development/zh-cn/backend/spi/registry.md
+++ b/development/zh-cn/backend/spi/registry.md
@@ -1,30 +1,26 @@
-### DolphinScheduler Registry SPI 主要设计
+### DolphinScheduler Registry SPI 扩展
#### 如何使用？
-首先你需要执行 `mvn -U install -Dmaven.test.skip=true` 安装插件，生成注册中心的插件 jar。目录是：dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/registry
+进行以下配置（以 zookeeper 为例）
-注意：**${VERSION}** 需要根据当前版本手动修改
+* 注册中心插件配置, 以Zookeeper 为例 (registry.properties)
+ dolphinscheduler-service/src/main/resources/registry.properties
+ ```registry.properties
+ registry.plugin.name=zookeeper
+ registry.servers=127.0.0.1:2181
+ ```
-其次进行以下配置（以 zookeeper 为例）
-
-|参数 |默认值| 描述|
-|--|--|--|
-registry.plugin.dir|./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/registry/zookeeper | 注册中心插件目录
-registry.plugin.name|zookeeper|注册中心具体插件名称
-registry.plugin.binding|registry|DolphinScheduler 插件类别
-registry.servers|127.0.0.1:2181|zk 连接地址
-
-具体配置信息请参考具体插件提供的参数信息，例如 zk：org/apache/dolphinscheduler/plugin/registry/zookeeper/ZookeeperConfiguration.java
+具体配置信息请参考具体插件提供的参数信息，例如 zk：`org/apache/dolphinscheduler/plugin/registry/zookeeper/ZookeeperConfiguration.java`
所有配置信息前缀需要 +registry，如 base.sleep.time.ms，在 registry 中应该这样配置：registry.base.sleep.time.ms=100
+#### 如何扩展
-#### FAQ
+`dolphinscheduler-registry-api` 定义了实现插件的标准，当你需要扩展插件的时候只需要实现 `org.apache.dolphinscheduler.registry.api.RegistryFactory` 即可。
-1: registry plugin not found
-
-请检查是否有执行 `mvn -U install -Dmaven.test.skip=true` ，此外，请检查配置文件中的 registry.plugin.dir 中配置的目录是否有相关插件。
+`dolphinscheduler-registry-plugin` 模块下是我们目前所提供的注册中心插件。
+#### FAQ
-2：registry connect timeout
+1：registry connect timeout
可以增加相关超时参数。
diff --git a/development/zh-cn/backend/spi/task.md b/development/zh-cn/backend/spi/task.md
index 1592527..488ce77 100644
--- a/development/zh-cn/backend/spi/task.md
+++ b/development/zh-cn/backend/spi/task.md
@@ -1,32 +1,12 @@
-## DolphinScheduler Task SPI 主要设计
-
-#### 开发环境如何使用？
-
-首先你需要执行 `mvn -U install -Dmaven.test.skip=true` 安装插件，生成注册中心的插件 jar。目录是：dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/task（版本号会跟随主版本号变更）
-
-注意：**${VERSION}** 需要根据当前版本手动修改
-
-执行此命令会生成所有模块默认提供的所有插件。
-
-其次在（dolphinscheduler-server/src/main/resources/worker.properties）进行配置插件目录
-```
-task.plugin.dir config the #task.plugin.dir config the Task Plugin dir . WorkerServer while find and load the Task Plugin Jar from this dir when deploy and start WorkerServer on the server .
-task.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/task
-```
-本地开发调试也可采用以下方式进行配置，例如：
-```
-maven.local.repository=/Users/localRepository
-#task.plugin.binding config the task plugin need be load when development and run in IDE
-task.plugin.binding=./dolphinscheduler-task-plugin/dolphinscheduler-task-shell/pom.xml
-```
-请根据实际情况进行配置。
+## DolphinScheduler Task SPI 扩展
#### 如何进行任务插件开发？
-### 后端开发：
-
org.apache.dolphinscheduler.spi.task.TaskChannel
插件实现以上接口即可。主要包含创建任务（任务初始化，任务运行等方法）、任务取消，如果是 yarn 任务，则需要实现 org.apache.dolphinscheduler.plugin.task.api.AbstractYarnTask。
我们在 dolphinscheduler-task-api 模块提供了所有任务对外访问的 API，而 dolphinscheduler-spi 模块则是 spi 通用代码库，定义了所有的插件模块，比如告警模块，注册中心模块等，你可以详细阅读查看。
+
+*NOTICE*
+由于任务插件设计到前端页面，目前前端的SPI还没有实现，因此你需要单独实现插件对应的前端页面。
diff --git a/development/zh-cn/development-environment-setup.md b/development/zh-cn/development-environment-setup.md
index 8015617..9671901 100644
--- a/development/zh-cn/development-environment-setup.md
+++ b/development/zh-cn/development-environment-setup.md
@@ -29,7 +29,6 @@ DolphinScheduler 开发环境配置有两个方式，分别是standalone模式
## DolphinScheduler Standalone快速开发模式
> **_注意：_** 仅供单机开发调试使用，默认使用 H2 Database,Zookeeper Testing Server
-> 如需测试插件，可自行修改 StandaloneServer中 `plugin.bind`，亦或修改配置文件，具体请查看插件说明
> Standalone 仅在 DolphinScheduler 1.3.9 及以后的版本支持
### 分支选择
@@ -41,12 +40,6 @@ DolphinScheduler 开发环境配置有两个方式，分别是standalone模式
### 启动后端
-编译后端相关依赖
-
-```shell
-mvn install -DskipTests
-```
-
在 Intellij IDEA 找到并启动类 `org.apache.dolphinscheduler.server.StandaloneServer` 即可完成后端启动
### 启动前端
@@ -93,32 +86,27 @@ DolphinScheduler 的元数据存储在关系型数据库中，目前支持的关
* 打开项目：使用开发工具打开项目，这里以 Intellij IDEA 为例，打开后需要一段时间，让 Intellij IDEA 完成以依赖的下载
-* 插件的配置（**仅 2.0 及以后的版本需要**）：编译对应的插件，在项目目录执行 `mvn -U clean install -Dmaven.test.skip=true` 完成注册插件的安装
-
- 注意：**${VERSION}** 需要根据当前版本手动修改，关于 ***.plugin.binding, maven.local.repository 则无需做任何修改
+* 插件的配置（**仅 2.0 及以后的版本需要**）：
- * 告警插件配置 (alert.properties)
- ```alert.properties
- alert.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/alert
- ```
- * 注册中心插件配置 (registry.properties)
+ * 注册中心插件配置, 以Zookeeper 为例 (registry.properties)
+ dolphinscheduler-service/src/main/resources/registry.properties
```registry.properties
- registry.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/registry/zookeeper
- ```
- * 任务插件配置 (worker.properties)
- ```worker.properties
- task.plugin.dir=./dolphinscheduler-dist/target/dolphinscheduler-dist-${VERSION}/lib/plugin/task
+ registry.plugin.name=zookeeper
+ registry.servers=127.0.0.1:2181
```
* 必要的修改
* 如果使用 MySQL 作为元数据库，需要先修改 `dolphinscheduler/pom.xml`，将 `mysql-connector-java` 依赖的 `scope` 改为 `compile`，使用 PostgreSQL 则不需要
- * 修改数据库配置，修改 `dolphinscheduler/dolphinscheduler-dao/datasource.properties` 文件中的数据库配置
-
- ```properties
- # 本样例以 MySQL 为例，其中数据库名为 dolphinscheduler，账户名密码均为 dolphinscheduler
- spring.datasource.driver-class-name=com.mysql.jdbc.Driver
- spring.datasource.url=jdbc:mysql://localhost:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8&allowMultiQueries=true
- spring.datasource.username=dolphinscheduler
- spring.datasource.password=dolphinscheduler
+ * 修改数据库配置，修改 `dolphinscheduler-dao/src/main/resources/application-mysql.yaml` 文件中的数据库配置
+
+
+ 本样例以 MySQL 为例，其中数据库名为 dolphinscheduler，账户名密码均为 dolphinscheduler
+ ```application-mysql.yaml
+ spring:
+ datasource:
+ driver-class-name: com.mysql.jdbc.Driver
+ url: jdbc:mysql://127.0.0.1:3306/dolphinscheduler?useUnicode=true&characterEncoding=UTF-8
+ username: ds_user
+ password: dolphinscheduler
```
* 修改日志级别：为以下配置增加一行内容 `<appender-ref ref="STDOUT"/>` 使日志能在命令行中显示
@@ -139,8 +127,6 @@ DolphinScheduler 的元数据存储在关系型数据库中，目前支持的关
</root>
```
-> **_注意：_** 上述准备工作中，插件的安装仅 DolphinScheduler 2.0 及以后的版本需要运行，2.0 之前的版本不需要运行该命令
-
##### 启动服务
我们需要启动三个必须服务，包括 MasterServer，WorkerServer，ApiApplicationServer，如果有需求可以启动可选服务 LoggerServer