You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by lg...@apache.org on 2020/02/11 12:01:48 UTC
[incubator-dolphinscheduler-website] branch master updated: add
upgrade doc and deployment doc in the default version
This is an automated email from the ASF dual-hosted git repository.
lgcareer pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-dolphinscheduler-website.git
The following commit(s) were added to refs/heads/master by this push:
new d5513bd add upgrade doc and deployment doc in the default version
new 18082c2 Merge pull request #87 from lgcareer/master
d5513bd is described below
commit d5513bd04e1cd3fbf91242e2ce9f58aae9d7ab6e
Author: lgcareer <18...@163.com>
AuthorDate: Tue Feb 11 19:57:13 2020 +0800
add upgrade doc and deployment doc in the default version
---
docs/en-us/development/release.md | 2 +-
docs/en-us/user_doc/deployment.md | 218 ++++++++++++++++++++++++++++++++++++++
docs/en-us/user_doc/upgrade.md | 42 ++++++++
3 files changed, 261 insertions(+), 1 deletion(-)
diff --git a/docs/en-us/development/release.md b/docs/en-us/development/release.md
index 72a7a29..dd68f0c 100644
--- a/docs/en-us/development/release.md
+++ b/docs/en-us/development/release.md
@@ -221,7 +221,7 @@ mkdir -p ~/ds_svn/dev/dolphinscheduler/${RELEASE.VERSION}
cd ~/ds_svn/dev/dolphinscheduler/${RELEASE.VERSION}
```
-Add source code packages, binary packages and executable binary packages of sharding-proxy to SVN working directory.
+Add source code packages, binary packages and executable binary packages to SVN working directory.
```shell
cp -f ~/incubator-dolphinscheduler/dolphinscheduler-dist/target/*.zip ~/ds_svn/dev/dolphinscheduler/${RELEASE.VERSION}
diff --git a/docs/en-us/user_doc/deployment.md b/docs/en-us/user_doc/deployment.md
new file mode 100644
index 0000000..332a12e
--- /dev/null
+++ b/docs/en-us/user_doc/deployment.md
@@ -0,0 +1,218 @@
+# Deployment Document
+
+There are two deployment modes:
+
+- automatic deployment
+- source code compile and then deployment
+
+## Preparations
+
+Download the latest version of the installation package, download address： [download](https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler/1.2.1/),
+download apache-dolphinscheduler-incubating-x.x.x-dolphinscheduler-bin.tar.gz
+
+
+
+#### Preparations 1: Installation of basic software (self-installation of required items)
+
+ * PostgreSQL (8.2.15+) or Mysql (5.5+) : You can choose either PostgreSQL or Mysql.
+ * JDK (1.8+) : Mandatory
+ * ZooKeeper(3.4.6+) ：Mandatory
+ * Hadoop (2.6+) or MinIo ：Optionally, if you need to use the resource upload function, You can choose either Hadoop or MinIo.
+ * Hive (1.2.1) : Optional, hive task submission needs to be installed
+ * Spark(1.x,2.x) : Optional, Spark task submission needs to be installed
+
+```
+ Note: DolphinScheduler itself does not rely on Hadoop, Hive, Spark, PostgreSQL, but only calls their Client to run the corresponding tasks.
+```
+
+#### Preparations 2: Create deployment users
+
+- Deployment users are created on all machines that require deployment scheduling, because the worker service executes jobs in `sudo-u {linux-user}`, so deployment users need sudo privileges and are confidential.
+
+```
+vi /etc/sudoers
+
+# For example, the deployment user is an dolphinscheduler account
+dolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL
+
+# And you need to comment out the Default requiretty line
+#Default requiretty
+```
+
+#### Preparations 3: SSH Secret-Free Configuration
+Configure SSH secret-free login on deployment machines and other installation machines. If you want to install dolphinscheduler on deployment machines, you need to configure native password-free login itself.
+
+- Connect the host and other machines SSH
+
+#### Preparations 4: database initialization
+
+* Create databases and accounts
+
+ Execute the following command to create database and account
+
+ ```
+ CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
+ GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'%' IDENTIFIED BY '{password}';
+ GRANT ALL PRIVILEGES ON dolphinscheduler.* TO '{user}'@'localhost' IDENTIFIED BY '{password}';
+ flush privileges;
+ ```
+
+* creates tables and imports basic data
+ Modify the following attributes in ./conf/application.properties
+
+ ```
+ spring.datasource.url
+ spring.datasource.username
+ spring.datasource.password
+ ```
+
+ Execute scripts for creating tables and importing basic data
+
+ ```
+ sh ./script/create-dolphinscheduler.sh
+ ```
+
+#### Preparations 5: Modify the deployment directory permissions and operation parameters
+
+ instruction of dolphinscheduler-bin directory
+
+```directory
+bin : Basic service startup script
+DISCLAIMER : DISCLAIMER
+conf : Project Profile
+lib : The project relies on jar packages, including individual module jars and third-party jars
+LICENSE : LICENSE
+licenses : licenses
+NOTICE : NOTICE
+script : Cluster Start, Stop and Service Monitor Start and Stop scripts
+sql : The project relies on SQL files
+ui : The ui files
+install.sh : One-click deployment script
+```
+
+- Modify permissions (please modify the 'deployUser' to the corresponding deployment user) so that the deployment user has operational privileges on the dolphinscheduler-bin directory
+
+ `sudo chown -R deployUser:deployUser dolphinscheduler-bin`
+
+- Modify the `.dolphinscheduler_env.sh` environment variable in the conf/env/directory
+
+- Modify deployment parameters (depending on your server and business situation):
+
+ - Modify the parameters in `install.sh` to replace the values required by your business
+ - MonitorServerState switch variable, added in version 1.0.3, controls whether to start the self-start script (monitor master, worker status, if off-line will start automatically). The default value of "false" means that the self-start script is not started, and if it needs to start, it is changed to "true".
+ - 'hdfsStartupSate' switch variable controls whether to start hdfs
+ The default value of "false" means not to start hdfs
+ Change the variable to 'true' if you want to use hdfs, you also need to create the hdfs root path by yourself, that 'hdfsPath' in install.sh.
+
+ - If you use hdfs-related functions, you need to copy**hdfs-site.xml** and **core-site.xml** to the conf directory
+
+
+## Deployment
+Either of the following two methods can be deployed,binary file deployment is recommended, and experienced partners can use source deployment as well.
+
+### Binary file Deployment
+
+- Switch to deployment user, one-click deployment
+
+ `sh install.sh`
+
+- Use the `jps` command to check if the services are started (`jps` comes from `Java JDK`)
+
+```aidl
+ MasterServer ----- Master Service
+ WorkerServer ----- Worker Service
+ LoggerServer ----- Logger Service
+ ApiApplicationServer ----- API Service
+ AlertServer ----- Alert Service
+```
+
+If all services are normal, the automatic deployment is successful
+
+
+After successful deployment, the log can be viewed and stored in a specified folder.
+
+```logPath
+ logs/
+ ├── dolphinscheduler-alert-server.log
+ ├── dolphinscheduler-master-server.log
+ |—— dolphinscheduler-worker-server.log
+ |—— dolphinscheduler-api-server.log
+ |—— dolphinscheduler-logger-server.log
+```
+
+### Compile source code to deploy
+
+After downloading the release version of the source package, unzip it into the root directory
+
+* Execute the compilation command：
+
+```
+ mvn -U clean package -Prelease -Dmaven.test.skip=true
+```
+
+* View directory
+
+After normal compilation, `apache-dolphinscheduler-incubating-${latest.release.version}-dolphinscheduler-bin.tar.gz`
+is generated in the `./dolphinscheduler-dist/target` directory
+
+- Switch to deployment user, one-click deployment
+
+ `sh install.sh`
+
+### Start-and-stop services commonly used in systems (for service purposes, please refer to System Architecture Design for details)
+
+* stop all services in the cluster
+
+ ` sh ./bin/stop-all.sh`
+
+* start all services in the cluster
+
+ ` sh ./bin/start-all.sh`
+
+* start and stop one master server
+
+```master
+sh ./bin/dolphinscheduler-daemon.sh start master-server
+sh ./bin/dolphinscheduler-daemon.sh stop master-server
+```
+
+* start and stop one worker server
+
+```worker
+sh ./bin/dolphinscheduler-daemon.sh start worker-server
+sh ./bin/dolphinscheduler-daemon.sh stop worker-server
+```
+
+* start and stop api server
+
+```Api
+sh ./bin/dolphinscheduler-daemon.sh start api-server
+sh ./bin/dolphinscheduler-daemon.sh stop api-server
+```
+* start and stop logger server
+
+```Logger
+sh ./bin/dolphinscheduler-daemon.sh start logger-server
+sh ./bin/dolphinscheduler-daemon.sh stop logger-server
+```
+* start and stop alert server
+
+```Alert
+sh ./bin/dolphinscheduler-daemon.sh start alert-server
+sh ./bin/dolphinscheduler-daemon.sh stop alert-server
+```
+
+## Database Upgrade
+Modify the following properties in ./conf/application.properties
+
+ ```
+ spring.datasource.url
+ spring.datasource.username
+ spring.datasource.password
+ ```
+The database can be upgraded automatically by executing the following command:
+```upgrade
+sh ./script/upgrade-dolphinscheduler.sh
+```
+
+
diff --git a/docs/en-us/user_doc/upgrade.md b/docs/en-us/user_doc/upgrade.md
new file mode 100644
index 0000000..6c7a29c
--- /dev/null
+++ b/docs/en-us/user_doc/upgrade.md
@@ -0,0 +1,42 @@
+
+# DolphinScheduler upgrade documentation
+
+## 1. Back up the previous version of the files and database
+
+## 2. Stop all services of dolphinscheduler
+
+ `sh ./script/stop-all.sh`
+
+## 3. Download the new version of the installation package
+
+- [download](https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler/1.2.1/), download the latest version of the installation packages
+- The following upgrade operations need to be performed in the new version of the directory
+
+## 4. Database upgrade
+- Modify the following properties in conf/application.properties
+
+```
+ spring.datasource.url
+ spring.datasource.username
+ spring.datasource.password
+```
+
+- Execute database upgrade script
+
+`sh ./script/upgrade-dolphinscheduler.sh`
+
+## 5. service upgrade
+you can choose either jetty deployment or nginx deployment.
+### jetty deployment
+- Modify the content of the install.sh configuration and execute the upgrade script
+
+ `sh install.sh`
+- Visit the following url, interface address (modify it yourself)
+ http://192.168.xx.xx:12345/dolphinscheduler
+
+### nginx deployment
+- Please download the source packages and execute `mvn clean package -Pnginx`
+- Overwrite the previous version of the dist directory
+- Restart the nginx service
+
+ `systemctl restart nginx`