You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by zh...@apache.org on 2022/05/26 06:56:44 UTC
[dolphinscheduler] branch 3.0.0-beta-prepare updated (040f3a838c -> 6539573a53)
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a change to branch 3.0.0-beta-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
from 040f3a838c [doc] Add setValue syntax in local param (#10245)
new 8ad17efeb5 [fix] Deploy scp-hosts use fullpath avoid dangling (#10249)
new 54c5e8040a [Bug] init docker compose DATABASE env (#10244)
new 6539573a53 [doc] Correct all setup documents (#10074)
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
deploy/docker/.env | 1 +
docs/docs/en/guide/installation/kubernetes.md | 2 +-
docs/docs/en/guide/installation/pseudo-cluster.md | 4 +-
docs/docs/en/guide/installation/standalone.md | 14 ++--
docs/docs/en/guide/start/quick-start.md | 2 +-
docs/docs/en/guide/upgrade.md | 66 +++++++++++--------
docs/docs/zh/guide/installation/kubernetes.md | 4 +-
docs/docs/zh/guide/installation/pseudo-cluster.md | 2 +-
docs/docs/zh/guide/installation/standalone.md | 13 ++--
docs/docs/zh/guide/start/quick-start.md | 2 +-
docs/docs/zh/guide/upgrade.md | 75 +++++++++++++---------
.../src/main/bin/upgrade-schema.sh | 4 +-
script/scp-hosts.sh | 4 +-
13 files changed, 112 insertions(+), 81 deletions(-)
[dolphinscheduler] 03/03: [doc] Correct all setup documents (#10074)
Posted by zh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch 3.0.0-beta-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 6539573a534ce9906b16f943d5c3c2ba3a3c4594
Author: Jiajie Zhong <zh...@hotmail.com>
AuthorDate: Thu May 26 14:53:47 2022 +0800
[doc] Correct all setup documents (#10074)
* Correct the login page URL
* change version and database configuration
* change setup and upgrade
(cherry picked from commit e3579dd13d54dc16ff8c4ddea3b8e9a2f3a767ad)
---
docs/docs/en/guide/installation/kubernetes.md | 2 +-
docs/docs/en/guide/installation/pseudo-cluster.md | 4 +-
docs/docs/en/guide/installation/standalone.md | 14 +++--
docs/docs/en/guide/start/quick-start.md | 2 +-
docs/docs/en/guide/upgrade.md | 66 ++++++++++++--------
docs/docs/zh/guide/installation/kubernetes.md | 4 +-
docs/docs/zh/guide/installation/pseudo-cluster.md | 2 +-
docs/docs/zh/guide/installation/standalone.md | 13 ++--
docs/docs/zh/guide/start/quick-start.md | 2 +-
docs/docs/zh/guide/upgrade.md | 75 +++++++++++++----------
10 files changed, 106 insertions(+), 78 deletions(-)
diff --git a/docs/docs/en/guide/installation/kubernetes.md b/docs/docs/en/guide/installation/kubernetes.md
index 4162b03217..9a05f24295 100644
--- a/docs/docs/en/guide/installation/kubernetes.md
+++ b/docs/docs/en/guide/installation/kubernetes.md
@@ -53,7 +53,7 @@ $ kubectl port-forward --address 0.0.0.0 -n test svc/dolphinscheduler-api 12345:
> **Tip**: If the error of `unable to do port forwarding: socat not found` appears, you need to install `socat` first.
-Access the web: `http://localhost:12345/dolphinscheduler` (Modify the IP address if needed).
+Access the web: `http://localhost:12345/dolphinscheduler/ui` (Modify the IP address if needed).
Or when `api.service.type=NodePort` you need to execute the command:
diff --git a/docs/docs/en/guide/installation/pseudo-cluster.md b/docs/docs/en/guide/installation/pseudo-cluster.md
index d6ff451175..a37e0b58e4 100644
--- a/docs/docs/en/guide/installation/pseudo-cluster.md
+++ b/docs/docs/en/guide/installation/pseudo-cluster.md
@@ -2,7 +2,7 @@
The purpose of the pseudo-cluster deployment is to deploy the DolphinScheduler service on a single machine. In this mode, DolphinScheduler's master, worker, API server, are all on the same machine.
-If you are a new hand and want to experience DolphinScheduler functions, we recommend you install follow [Standalone deployment](standalone.md). If you want to experience more complete functions and schedule massive tasks, we recommend you install follow [pseudo-cluster deployment](pseudo-cluster.md). If you want to deploy DolphinScheduler in production, we recommend you follow [cluster deployment](cluster.md) or [Kubernetes deployment](kubernetes.md).
+If you are a new hand and want to experience DolphinScheduler functions, we recommend you install follow [Standalone deployment](standalone.md). If you want to experience more complete functions and schedule massive tasks, we recommend you install follow[pseudo-cluster deployment. If you want to deploy DolphinScheduler in production, we recommend you follow [cluster deployment](cluster.md) or [Kubernetes deployment](kubernetes.md).
## Preparation
@@ -204,7 +204,7 @@ sh ./bin/install.sh
## Login DolphinScheduler
-Access address `http://localhost:12345/dolphinscheduler` and login DolphinScheduler UI. The default username and password are **admin/dolphinscheduler123**
+Access address `http://localhost:12345/dolphinscheduler/ui` and login DolphinScheduler UI. The default username and password are **admin/dolphinscheduler123**
## Start or Stop Server
diff --git a/docs/docs/en/guide/installation/standalone.md b/docs/docs/en/guide/installation/standalone.md
index 80406bd30d..9ef53f3808 100644
--- a/docs/docs/en/guide/installation/standalone.md
+++ b/docs/docs/en/guide/installation/standalone.md
@@ -2,7 +2,7 @@
Standalone only for quick experience for DolphinScheduler.
-If you are a new hand and want to experience DolphinScheduler functions, we recommend you install follow [Standalone deployment](standalone.md). If you want to experience more complete functions and schedule massive tasks, we recommend you install follow [pseudo-cluster deployment](pseudo-cluster.md). If you want to deploy DolphinScheduler in production, we recommend you follow [cluster deployment](cluster.md) or [Kubernetes deployment](kubernetes.md).
+If you are a new hand and want to experience DolphinScheduler functions, we recommend you install follow Standalone deployment. If you want to experience more complete functions and schedule massive tasks, we recommend you install follow [pseudo-cluster deployment](pseudo-cluster.md). If you want to deploy DolphinScheduler in production, we recommend you follow [cluster deployment](cluster.md) or [Kubernetes deployment](kubernetes.md).
> **_Note:_** Standalone only recommends the usage of fewer than 20 workflows, because it uses in-memory H2 Database in default, ZooKeeper Testing Server, too many tasks may cause instability.
> When Standalone stops or restarts, in-memory H2 database will clear up. To use Standalone with external databases like mysql or postgresql, please see [`Database Configuration`](#database-configuration).
@@ -27,7 +27,7 @@ sh ./bin/dolphinscheduler-daemon.sh start standalone-server
### Login DolphinScheduler
-Access address `http://localhost:12345/dolphinscheduler` and login DolphinScheduler UI. The default username and password are **admin/dolphinscheduler123**
+Access address `http://localhost:12345/dolphinscheduler/ui` and login DolphinScheduler UI. The default username and password are **admin/dolphinscheduler123**
### Start or Stop Server
@@ -46,11 +46,14 @@ sh ./bin/dolphinscheduler-daemon.sh stop standalone-server
[jdk]: https://www.oracle.com/technetwork/java/javase/downloads/index.html
-### Database Configuration
+## Database Configuration
+
+Standalone server use H2 database as its metadata store, it is easy and users do not need to start database before they set up server.
+But if user want to store metabase in other database like MySQL or PostgreSQL, they have to change some configuration. And we here use
+MySQL as an example to illustrate how to configure an external database:
-* Use mysql as an example to illustrate how to configure an external database:
* First of all, follow the instructions in [pseudo-cluster deployment](pseudo-cluster.md) `Initialize the Database` section to create and initialize database
-* Set the following environment variables in your terminal with your database username and password for {user} and {password}:
+* Set the following environment variables in your terminal with your database username and password for `{user}` and `{password}`:
```shell
export DATABASE=${DATABASE:-mysql}
@@ -60,5 +63,4 @@ export SPRING_DATASOURCE_PASSWORD={password}
```
* Add mysql-connector-java driver to `./standalone-server/libs/standalone-server/`, see [pseudo-cluster deployment](pseudo-cluster.md) `Initialize the Database` section about where to download
-
* Start standalone-server, now you are using mysql as database and it will not clear up your data when you stop or restart standalone-server.
diff --git a/docs/docs/en/guide/start/quick-start.md b/docs/docs/en/guide/start/quick-start.md
index 9310f9d118..13f9c5a7b0 100644
--- a/docs/docs/en/guide/start/quick-start.md
+++ b/docs/docs/en/guide/start/quick-start.md
@@ -6,7 +6,7 @@
* Administrator user login
- > Address：http://localhost:12345/dolphinscheduler Username and password: `admin/dolphinscheduler123`
+ > Address：http://localhost:12345/dolphinscheduler/ui Username and password: `admin/dolphinscheduler123`
diff --git a/docs/docs/en/guide/upgrade.md b/docs/docs/en/guide/upgrade.md
index f3d3f4c779..be901a6a50 100644
--- a/docs/docs/en/guide/upgrade.md
+++ b/docs/docs/en/guide/upgrade.md
@@ -1,19 +1,28 @@
-# DolphinScheduler Upgrade Documentation
+# DolphinScheduler Upgrade
-## Back-Up Previous Version's Files and Database
+## Prepare
-## Stop All Services of DolphinScheduler
+### Backup Previous Version's Files and Database
- `sh ./script/stop-all.sh`
+To prevent data loss by some miss-operation, it is recommended to back up data before upgrading. The backup way according to your environment.
-## Download the Latest Version Installation Package
+### Download the Latest Version Installation Package
-- [download](/en-us/download/download.html) the latest version of the installation packages.
-- The following upgrade operations need to be performed in the new version's directory.
+Download the latest binary distribute package from [download](/en-us/download/download.html) and then put it in the different
+directory where current service running. And all below command is running in this directory.
-## Database Upgrade
+## Upgrade
-- If using MySQL as the database to run DolphinScheduler, please config it in `./bin/env/dolphinscheduler_env.sh`, change username and password to yours, and add MYSQL connector jar into lib dir `./tools/libs`, here we download `mysql-connector-java-8.0.16.jar`, and then correctly configure database connection information. You can download MYSQL connector jar from [here](https://downloads.MySQL.com/archives/c-j/). Otherwise, PostgreSQL is the default database.
+### Stop All Services of DolphinScheduler
+
+Stop all services of dolphinscheduler according to your deployment method. If you deploy your dolphinscheduler according to [cluster deployment](./installation/cluster.md), you can stop all services by command `sh ./script/stop-all.sh`.
+
+### Upgrade Database
+
+Change configuration in `./bin/env/dolphinscheduler_env.sh` ({user} and {password} are changed to your database username and password), and then run the upgrade script.
+
+Using MySQL as an example, change the value if you use other databases. Please manually download the [mysql-connector-java driver jar](https://downloads.MySQL.com/archives/c-j/)
+jar package and add it to the `./tools/libs` directory, then change `./bin/ env/dolphinscheduler_env.sh` file
```shell
export DATABASE=${DATABASE:-mysql}
@@ -24,36 +33,39 @@
export SPRING_DATASOURCE_PASSWORD={password}
```
-- Execute database upgrade script:
+Execute database upgrade script: `sh ./tools/bin/upgrade-schema.sh`
+K
+### Upgrade Service
- `sh ./tools/bin/upgrade-schema.sh`
+#### Change Configuration `bin/env/install_config.conf`
-## Backend Service Upgrade
+- If you deploy with Pseudo-Cluster deployment, change it according to [Pseudo-Cluster](./installation/pseudo-cluster.md) section "Modify Configuration".
+- If you deploy with Cluster deployment, change it according to [Cluster](./installation/cluster.md) section "Modify Configuration".
-### Modify the Content in `conf/config/install_config.conf` File
+And them run command `sh ./bin/start-all.sh` to start all services.
-- Standalone Deployment please refer to the [Standalone-Deployment](./installation/standalone.md).
-- Cluster Deployment please refer to the [Cluster-Deployment](./installation/cluster.md).
+## Notice
-#### Masters Need Attentions
+### Differences of worker group (before or after version 1.3.1 of dolphinscheduler)
-Create worker group in 1.3.1 version has a different design:
+The architecture of worker group is different between version before version 1.3.1 until version 2.0.0
-- Before version 1.3.1 worker group can be created through UI interface.
-- Since version 1.3.1 worker group can be created by modifying the worker configuration.
+- Before version 1.3.1(include itself) worker group can be created through UI interface.
+- Since version 1.3.1 and before version 2.0.0, worker group can be created by modifying the worker configuration.
-#### When Upgrade from Version Before 1.3.1 to 1.3.2, the Below Operations are What We Need to Do to Keep Worker Group Configuration Consist with Previous
+#### How Can I Do When I Upgrade from 1.3.1 to version before 2.0.0
-1. Go to the backup database, search records in `t_ds_worker_group table`, mainly focus `id, name and IP` three columns.
+* Check the backup database, search records in table `t_ds_worker_group` table and mainly focus on three columns: `id, name and IP`.
| id | name | ip_list |
| :--- | :---: | ---: |
| 1 | service1 | 192.168.xx.10 |
| 2 | service2 | 192.168.xx.11,192.168.xx.12 |
-2. Modify the worker configuration in `conf/config/install_config.conf` file.
+* Modify worker related configuration in `bin/env/install_config.conf`.
Assume bellow are the machine worker service to be deployed:
+
| hostname | ip |
| :--- | :---: |
| ds1 | 192.168.xx.10 |
@@ -69,12 +81,12 @@ workers="ds1:service1,ds2:service2,ds3:service2"
#### The Worker Group has Been Enhanced in Version 1.3.2
-Workers in 1.3.1 can't belong to more than one worker group, but in 1.3.2 it's supported. So in 1.3.1 it's not supported when `workers="ds1:service1,ds1:service2"`, and in 1.3.2 it's supported.
-
-### Execute Deploy Script
+Workers in 1.3.1 can only belong to one worker group, but after version 1.3.2 and before version 2.0.0 worker support more than one worker group.
-```shell
-`sh install.sh`
+```sh
+workers="ds1:service1,ds1:service2"
```
+#### Recovery UI Create Worker Group after Version 2.0.0
+After version 2.0.0, include itself, we are recovery function create worker group from web UI.
diff --git a/docs/docs/zh/guide/installation/kubernetes.md b/docs/docs/zh/guide/installation/kubernetes.md
index 569fdb0d00..bc058e951f 100644
--- a/docs/docs/zh/guide/installation/kubernetes.md
+++ b/docs/docs/zh/guide/installation/kubernetes.md
@@ -54,7 +54,7 @@ $ kubectl port-forward --address 0.0.0.0 -n test svc/dolphinscheduler-api 12345:
> **提示**: 如果出现 `unable to do port forwarding: socat not found` 错误, 需要先安装 `socat`
-访问前端页面：http://localhost:12345/dolphinscheduler，如果有需要请修改成对应的 IP 地址
+访问前端页面：http://localhost:12345/dolphinscheduler/ui，如果有需要请修改成对应的 IP 地址
或者当 `api.service.type=NodePort` 时，你需要执行命令：
@@ -64,7 +64,7 @@ NODE_PORT=$(kubectl get svc {{ template "dolphinscheduler.fullname" . }}-api -n
echo http://$NODE_IP:$NODE_PORT/dolphinscheduler
```
-然后访问前端页面: http://localhost:12345/dolphinscheduler
+然后访问前端页面: http://localhost:12345/dolphinscheduler/ui
默认的用户是`admin`，默认的密码是`dolphinscheduler123`
diff --git a/docs/docs/zh/guide/installation/pseudo-cluster.md b/docs/docs/zh/guide/installation/pseudo-cluster.md
index 107d27b2f1..f76b834adc 100644
--- a/docs/docs/zh/guide/installation/pseudo-cluster.md
+++ b/docs/docs/zh/guide/installation/pseudo-cluster.md
@@ -200,7 +200,7 @@ sh ./bin/install.sh
## 登录 DolphinScheduler
-浏览器访问地址 http://localhost:12345/dolphinscheduler 即可登录系统UI。默认的用户名和密码是 **admin/dolphinscheduler123**
+浏览器访问地址 http://localhost:12345/dolphinscheduler/ui 即可登录系统UI。默认的用户名和密码是 **admin/dolphinscheduler123**
## 启停服务
diff --git a/docs/docs/zh/guide/installation/standalone.md b/docs/docs/zh/guide/installation/standalone.md
index ac511e8b80..074756a2dc 100644
--- a/docs/docs/zh/guide/installation/standalone.md
+++ b/docs/docs/zh/guide/installation/standalone.md
@@ -2,7 +2,7 @@
Standalone 仅适用于 DolphinScheduler 的快速体验.
-如果你是新手，想要体验 DolphinScheduler 的功能，推荐使用[Standalone](standalone.md)方式体检。如果你想体验更完整的功能，或者更大的任务量，推荐使用[伪集群部署](pseudo-cluster.md)。如果你是在生产中使用，推荐使用[集群部署](cluster.md)或者[kubernetes](kubernetes.md)
+如果你是新手，想要体验 DolphinScheduler 的功能，推荐使用Standalone方式体检。如果你想体验更完整的功能，或者更大的任务量，推荐使用[伪集群部署](pseudo-cluster.md)。如果你是在生产中使用，推荐使用[集群部署](cluster.md)或者[kubernetes](kubernetes.md)
> **_注意:_** Standalone仅建议20个以下工作流使用，因为其采用内存式的H2 Database, Zookeeper Testing Server，任务过多可能导致不稳定，并且如果重启或者停止standalone-server会导致内存中数据库里的数据清空。
> 如果您要连接外部数据库，比如mysql或者postgresql，请看[配置数据库](#配置数据库)
@@ -27,7 +27,7 @@ sh ./bin/dolphinscheduler-daemon.sh start standalone-server
### 登录 DolphinScheduler
-浏览器访问地址 http://localhost:12345/dolphinscheduler 即可登录系统UI。默认的用户名和密码是 **admin/dolphinscheduler123**
+浏览器访问地址 http://localhost:12345/dolphinscheduler/ui 即可登录系统UI。默认的用户名和密码是 **admin/dolphinscheduler123**
## 启停服务
@@ -42,11 +42,13 @@ sh ./bin/dolphinscheduler-daemon.sh stop standalone-server
[jdk]: https://www.oracle.com/technetwork/java/javase/downloads/index.html
-### 配置数据库
+## 配置数据库
+
+Standalone server 使用 H2 数据库作为其元数据存储数据，这是为了上手简单，用户在启动服务器之前不需要启动数据库。但是如果用户想将元数据库存储在
+MySQL 或 PostgreSQL 等其他数据库中，他们必须更改一些配置。我们这里以 MySQL 为例来说明如何配置外部数据库：
-* 这里以mysql为例介绍如何连接外部数据库
* 首先，参照 [伪集群部署](pseudo-cluster.md) `初始化数据库` 创建并初始化数据库
-* 在你的命令行设定下列环境变量，将{user}和{password}改为你数据库的用户名和密码
+* 在你的命令行设定下列环境变量，将 `{user}` 和 `{password}` 改为你数据库的用户名和密码
```shell
export DATABASE=${DATABASE:-mysql}
@@ -56,5 +58,4 @@ export SPRING_DATASOURCE_PASSWORD={password}
```
* 将mysql-connector-java驱动加到`./standalone-server/libs/standalone-server/`目录下, 下载方法见 [伪集群部署](pseudo-cluster.md) `初始化数据库` 一栏
-
* 启动standalone-server，此时你已经连接上mysql，重启或者停止standalone-server并不会清空您数据库里的数据
diff --git a/docs/docs/zh/guide/start/quick-start.md b/docs/docs/zh/guide/start/quick-start.md
index a40740b846..5de08cc607 100644
--- a/docs/docs/zh/guide/start/quick-start.md
+++ b/docs/docs/zh/guide/start/quick-start.md
@@ -4,7 +4,7 @@
* 管理员用户登录
- >地址：http://localhost:12345/dolphinscheduler 用户名/密码：admin/dolphinscheduler123
+ >地址：http://localhost:12345/dolphinscheduler/ui 用户名/密码：admin/dolphinscheduler123
diff --git a/docs/docs/zh/guide/upgrade.md b/docs/docs/zh/guide/upgrade.md
index dc1d6cb732..763d9ac963 100644
--- a/docs/docs/zh/guide/upgrade.md
+++ b/docs/docs/zh/guide/upgrade.md
@@ -1,20 +1,27 @@
+# DolphinScheduler 升级
-# DolphinScheduler升级文档
+## 准备工作
-## 1. 备份上一版本文件和数据库
+### 备份上一版本文件和数据库
-## 2. 停止dolphinscheduler所有服务
+为了防止操作错误导致数据丢失，建议升级之前备份数据，备份方法请结合你数据库的情况来定
- `sh ./script/stop-all.sh`
+### 下载新版本的安装包
-## 3. 下载新版本的安装包
+在[下载](/zh-cn/download/download.html)页面下载最新版本的二进制安装包，并将二进制包放到与当前 dolphinscheduler 服务不一样的路径中，以下升级操作都需要在新版本的目录进行。
-- [下载](/zh-cn/download/download.html), 下载最新版本的二进制安装包
-- 以下升级操作都需要在新版本的目录进行
+## 升级步骤
-## 4. 数据库升级
+### 停止 dolphinscheduler 所有服务
-- 如果选择 MySQL，请修改`./bin/env/dolphinscheduler_env.sh`中的如下配置（{user}和{password}改成你数据库的用户名和密码）, 还需要手动添加 [ mysql-connector-java 驱动 jar ](https://downloads.MySQL.com/archives/c-j/) 包到 lib 目录（`./tools/lib`）下，这里下载的是mysql-connector-java-8.0.16.jar
+根据你部署方式停止 dolphinscheduler 的所有服务，如果你是通过 [集群部署](./installation/cluster.md) 来部署你的 dolphinscheduler 的话，可以通过 `sh ./script/stop-all.sh` 停止全部服务。
+
+### 数据库升级
+
+修改 `./bin/env/dolphinscheduler_env.sh` 中的如下配置（{user}和{password}改成你数据库的用户名和密码），然后运行升级脚本。
+
+下面以 MySQL 为例，别的数据库仅需要修改成对应的配置即可。请先手动下载 [mysql-connector-java 驱动 jar](https://downloads.MySQL.com/archives/c-j/)
+jar 包 并添加到 `./tools/libs` 目录下，修改 `./bin/env/dolphinscheduler_env.sh` 文件
```shell
export DATABASE=${DATABASE:-mysql}
@@ -25,32 +32,38 @@
export SPRING_DATASOURCE_PASSWORD={password}
```
-- 执行数据库升级脚本
+执行数据库升级脚本：`sh ./tools/bin/upgrade-schema.sh`
+
+### 服务升级
+
+#### 修改 `bin/env/install_config.conf` 配置内容
-`sh ./tools/bin/upgrade-schema.sh`
+- 伪集群部署请参照[伪集群部署(Pseudo-Cluster)](./installation/pseudo-cluster.md)中的 `修改相关配置`
+- 集群部署请参照[集群部署(Cluster)](./installation/cluster.md)中的 `修改相关配置`
-## 5. 服务升级
+然后运行命令 `sh ./bin/start-all.sh` 重启全部服务。
-### 5.1 修改`conf/config/install_config.conf`配置内容
-单机部署请参照[单机部署(Standalone)](./installation/standalone.md)中的`6.修改运行参数部分`
-集群部署请参照[集群部署(Cluster)](./installation/cluster.md)中的`6.修改运行参数部分`
+## 注意事项
-### 注意事项
-创建worker分组在1.3.1版本和之前版本有了不同的设计
+### worker 分组的区别（以 1.3.1 版本为界）
+
+创建worker分组在1.3.1版本之前，与 1.3.1之后到 2.0.0 之间的版本有不同的设计：
- worker分组在1.3.1版本之前是通过UI界面创建
-- worker分组在1.3.1版本是修改worker配置指定
+- worker分组在1.3.1 到 2.0.0之前的版本是修改 worker 配置指定
+
+#### 面对这种区别我应该怎么升级
-### 1.3.1之前的版本升级1.3.2时如何设置worker分组与之前一致
+1.3.1之前的版本升级1.3.2时如何设置worker分组与之前一致
-1、查询已备份的数据库,查看t_ds_worker_group表记录，重点看下id、name和ip_list三个字段
+* 查询已备份的数据库，查看 `t_ds_worker_group` 表记录，重点看下id、name和ip_list三个字段
| id | name | ip_list |
| :--- | :---: | ---: |
| 1 | service1 | 192.168.xx.10 |
| 2 | service2 | 192.168.xx.11,192.168.xx.12 |
-2、修改conf/config/install_config.conf中的workers参数
+* 修改 `bin/env/install_config.conf` 中的 workers 参数
假设以下为要部署的worker主机名和ip的对应关系
| 主机名 | ip |
@@ -61,19 +74,19 @@
那么为了保持与之前版本worker分组一致，则需要把workers参数改为如下
-```shell
-#worker服务部署在哪台机器上,并指定此worker属于哪一个worker组
+```sh
+# worker服务部署在哪台机器上,并指定此worker属于哪一个worker组
workers="ds1:service1,ds2:service2,ds3:service2"
```
-### 1.3.2的worker分组进行了增强
-1.3.1版本的worker不能同时属于多个worker分组，1.3.2是可以支持的
-所以在1.3.1里面的workers="ds1:service1,ds1:service2"是不支持的，
-在1.3.2可以设置workers="ds1:service1,ds1:service2"
-
-### 5.2 执行部署脚本
-```shell
-`sh install.sh`
+#### 1.3.2及以后的版本对 worker 分组功能进行增强
+
+1.3.1 以及之前的版本worker不能同时属于多个worker分组，1.3.2及之后，2.0.0之前的版本是可以支持的，所以可以使用如下配置对一台worker配置多个分组
+
+```sh
+workers="ds1:service1,ds1:service2"
```
+#### 在 2.0.0 版本之后恢复 UI 创建 worker group
+在 2.0.0 以及之后的版本，我们恢复了在 UI 创建 worker group 的功能。
[dolphinscheduler] 02/03: [Bug] init docker compose DATABASE env (#10244)
Posted by zh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch 3.0.0-beta-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 54c5e8040a32e10d95cca214707701949f39f30e
Author: caishunfeng <ca...@gmail.com>
AuthorDate: Thu May 26 13:50:59 2022 +0800
[Bug] init docker compose DATABASE env (#10244)
skip source env if in docker
(cherry picked from commit 4351148f36b2ab63303d95a26e8f3330eb3c74e3)
---
deploy/docker/.env | 1 +
dolphinscheduler-tools/src/main/bin/upgrade-schema.sh | 4 +++-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/deploy/docker/.env b/deploy/docker/.env
index 52094a8b51..04b44500b4 100755
--- a/deploy/docker/.env
+++ b/deploy/docker/.env
@@ -19,5 +19,6 @@ HUB=apache
TAG=3.0.0-beta-1
TZ=Asia/Shanghai
+DATABASE=postgresql
SPRING_DATASOURCE_URL=jdbc:postgresql://dolphinscheduler-postgresql:5432/dolphinscheduler
REGISTRY_ZOOKEEPER_CONNECT_STRING=dolphinscheduler-zookeeper:2181
diff --git a/dolphinscheduler-tools/src/main/bin/upgrade-schema.sh b/dolphinscheduler-tools/src/main/bin/upgrade-schema.sh
index 055157bc2b..0541c35c4d 100755
--- a/dolphinscheduler-tools/src/main/bin/upgrade-schema.sh
+++ b/dolphinscheduler-tools/src/main/bin/upgrade-schema.sh
@@ -19,7 +19,9 @@
BIN_DIR=$(dirname $0)
DOLPHINSCHEDULER_HOME=${DOLPHINSCHEDULER_HOME:-$(cd $BIN_DIR/../..; pwd)}
-source "$DOLPHINSCHEDULER_HOME/bin/env/dolphinscheduler_env.sh"
+if [ "$DOCKER" != "true" ]; then
+ source "$DOLPHINSCHEDULER_HOME/bin/env/dolphinscheduler_env.sh"
+fi
JAVA_OPTS=${JAVA_OPTS:-"-server -Duser.timezone=${SPRING_JACKSON_TIME_ZONE} -Xms1g -Xmx1g -Xmn512m -XX:+PrintGCDetails -Xloggc:gc.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=dump.hprof"}
[dolphinscheduler] 01/03: [fix] Deploy scp-hosts use fullpath avoid dangling (#10249)
Posted by zh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch 3.0.0-beta-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 8ad17efeb56ad35d3a5252bcb0b7993c768b9c12
Author: Jiajie Zhong <zh...@hotmail.com>
AuthorDate: Thu May 26 09:59:40 2022 +0800
[fix] Deploy scp-hosts use fullpath avoid dangling (#10249)
This path fix error modify worker application.yaml when
running `scp-hosts.sh` script in different directory. Currently
setting only work when running `scp-hosts.sh` in path
`<DOLPHINSCHEDULER_HOME>/bin` using command `./scp-hosts.sh`.
And will fail when we run script in directory
`<DOLPHINSCHEDULER_HOME>` using command `./bin/scp-hosts.sh`
ref: #10209 and #10208
(cherry picked from commit 58afe5d8b5a76460bb058eec1c15447a4d6ba7be)
---
script/scp-hosts.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/script/scp-hosts.sh b/script/scp-hosts.sh
index 25b688170a..285f3280b2 100755
--- a/script/scp-hosts.sh
+++ b/script/scp-hosts.sh
@@ -47,7 +47,7 @@ do
fi
done
# set worker groups in application.yaml
- [[ -n ${workerIndex} ]] && sed -i "s/- default/- ${groupNames[$workerIndex]}/" ../worker-server/conf/application.yaml
+ [[ -n ${workerIndex} ]] && sed -i "s/- default/- ${groupNames[$workerIndex]}/" $workDir/../worker-server/conf/application.yaml
for dsDir in bin master-server worker-server alert-server api-server ui tools
do
@@ -56,7 +56,7 @@ do
scp -q -P $sshPort -r $workDir/../$dsDir $host:$installPath
done
# restore worker groups to default
- [[ -n ${workerIndex} ]] && sed -i "s/- ${groupNames[$workerIndex]}/- default/" ../worker-server/conf/application.yaml
+ [[ -n ${workerIndex} ]] && sed -i "s/- ${groupNames[$workerIndex]}/- default/" $workDir/../worker-server/conf/application.yaml
echo "scp dirs to $host/$installPath complete"
done