You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by gi...@apache.org on 2020/07/14 00:46:05 UTC
[incubator-dolphinscheduler-website] branch asf-site updated:
Automated deployment: Tue Jul 14 00:45:54 UTC 2020
74350fb6e7c9875d843eea52f779515bd9f62383
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/incubator-dolphinscheduler-website.git
The following commit(s) were added to refs/heads/asf-site by this push:
new b6ea12e Automated deployment: Tue Jul 14 00:45:54 UTC 2020 74350fb6e7c9875d843eea52f779515bd9f62383
b6ea12e is described below
commit b6ea12e44f51f749de158e40084e4a4163afd9d8
Author: dailidong <da...@users.noreply.github.com>
AuthorDate: Tue Jul 14 00:45:55 2020 +0000
Automated deployment: Tue Jul 14 00:45:54 UTC 2020 74350fb6e7c9875d843eea52f779515bd9f62383
---
.../development/development-environment-setup.html | 239 ++++++++++-----------
.../development/development-environment-setup.json | 2 +-
2 files changed, 112 insertions(+), 129 deletions(-)
diff --git a/zh-cn/docs/development/development-environment-setup.html b/zh-cn/docs/development/development-environment-setup.html
index d352271..e90dee7 100644
--- a/zh-cn/docs/development/development-environment-setup.html
+++ b/zh-cn/docs/development/development-environment-setup.html
@@ -18,14 +18,14 @@
<p>首先从远端仓库fork <a href="https://github.com/apache/incubator-dolphinscheduler">dolphinscheduler</a> 一份代码到自己的仓库中</p>
</li>
<li>
-<p>安装好MySQL/PostgreSQL,Zookeeper,Nginx,Hadoop(可选)</p>
+<p>在开发环境中安装好MySQL/PostgreSQL、JDK、MAVEN</p>
</li>
<li>
<p>把自己仓库clone到本地</p>
<p><code>git clone https://github.com/apache/incubator-dolphinscheduler.git</code></p>
</li>
<li>
-<p>git clone项目后，进入目录，执行以下命令。</p>
+<p>git clone项目后，进入项目目录，执行以下命令。</p>
</li>
</ol>
<pre><code>1. git branch -a #查看分支
@@ -33,148 +33,131 @@
3. git pull #同步分支
4. mvn -U clean package -Prelease -Dmaven.test.skip=true #由于项目使用了gRPC，所以需要先编译项目生成需要的类。
</code></pre>
+<h4>安装node</h4>
+<ol>
+<li>安装nvm<br>
+curl -o- <a href="https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh">https://raw.githubusercontent.com/nvm-sh/nvm/v0.35.3/install.sh</a> | bash</li>
+<li>刷新环境变量<br>
+source ~/.bash_profile</li>
+<li>安装node<br>
+nvm install v12.12.0<br>
+备注:mac用户还可以通过brew安装npm:brew install npm</li>
+<li>验证node安装成功<br>
+node --version</li>
+</ol>
+<h4>安装zookeeper</h4>
+<ol>
+<li>下载zookeeper<br>
+<a href="http://apache.mirrors.hoobly.com/zookeeper/zookeeper-3.6.1/apache-zookeeper-3.6.1-bin.tar.gz">http://apache.mirrors.hoobly.com/zookeeper/zookeeper-3.6.1/apache-zookeeper-3.6.1-bin.tar.gz</a></li>
+<li>复制配置文件<br>
+cp conf/zoo_sample.cfg conf/zoo.cfg</li>
+<li>修改配置<br>
+vi conf/zoo.cfg<br>
+dataDir=./tmp/zookeeper</li>
+<li>启动/停止zookeeper<br>
+./bin/zkServer.sh start
+./bin/zkServer.sh stop</li>
+</ol>
+<h4>创建数据库</h4>
+<ol>
+<li>创建用户名为ds_user,密码为dolphinscheduler的用户</li>
+</ol>
+<pre><code>mysql> CREATE DATABASE dolphinscheduler DEFAULT CHARACTER SET utf8 DEFAULT COLLATE utf8_general_ci;
+mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'ds_user'@'%' IDENTIFIED BY 'dolphinscheduler';
+mysql> GRANT ALL PRIVILEGES ON dolphinscheduler.* TO 'ds_user'@'localhost' IDENTIFIED BY 'dolphinscheduler';
+mysql> flush privileges;
+
+</code></pre>
+<h4>搭建前端</h4>
+<ol>
+<li>进入dolphinscheduler-ui的目录<br>
+cd dolphinscheduler-ui</li>
+<li>执行npm install</li>
+</ol>
<h4>搭建后端</h4>
<ol>
<li>
-<p>创建ds数据库CREATE DATABASE dolphinscheduler。</p>
+<p>将项目导入到idea中<br>
+file-->open</p>
</li>
<li>
-<p>创建表和初始化数据：
-修改dao模块resource目录下application.properties文件中的数据库配置信息，如果你使用MySQL作为数据库，那么还需要配置pom.xml中mysql-connector-java依赖的scope为compile，然后执行org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler的类，运行完，刷新数据库，表和数据都有了。</p>
+<p>修改dao模块resource目录下datasource.properties文件中的数据库配置信息</p>
+<pre><code> spring.datasource.driver-class-name=com.mysql.jdbc.Driver
+ spring.datasource.url=jdbc:mysql://localhost:3306/dolphinscheduler
+ spring.datasource.username=ds_user
+ spring.datasource.password=dolphinscheduler
+</code></pre>
</li>
<li>
-<p>启动MasterServer
-org.apache.dolphinscheduler.server.master.MasterServer类main函数增加如下代码：</p>
-<pre><code>System.setProperty("spring.profiles.active","master");
-</code></pre>
-<p>修改server模块resources目录下master_logback.xml文件，增加以下代码：</p>
-<pre><code><root level="INFO">
- <appender-ref ref="MASTERLOGFILE"/>
- <!-- 增加日志到控制台-->
- <appender-ref ref="STDOUT"/>
-</root>
-</code></pre>
-<p>修改common模块，quartz.properties中数据库配置信息，zookeeper.properties中链接信息(zookeeper.quorum), 然后执行MasterServer即可。</p>
+<p>修改根项目中pom.xml，将mysql-connector-java依赖的scope修改为compile</p>
</li>
<li>
-<p>启动WorkerServer
-org.apache.dolphinscheduler.server.worker.WorkerServer类main函数增加如下代码：</p>
-<pre><code>System.setProperty("spring.profiles.active","worker");
-</code></pre>
-<p>修改server模块resources目录下worker_logback.xml文件，增加以下代码：</p>
-<pre><code><root level="INFO">
- <appender-ref ref="TASKLOGFILE"/>
- <appender-ref ref="WORKERLOGFILE"/>
- <!-- 增加日志到控制台-->
- <appender-ref ref="STDOUT"/>
-</root>
-</code></pre>
-<p>然后执行MasterServer即可。</p>
+<p>刷新dao模块，运行org.apache.dolphinscheduler.dao.upgrade.shell.CreateDolphinScheduler的main方法，自动插入项目所需的表和数据</p>
+</li>
+<li>
+<p>修改service模块zookeeper.properties中链接信息(zookeeper.quorum)<br>
+zookeeper.quorum=localhost:2181</p>
+</li>
+<li>
+<p>修改dolphinscheduler-ui模块的.env文件</p>
</li>
</ol>
-<h4>搭建前端</h4>
-<p>进入dolphinscheduler-ui的目录，执行ui项目的编译，由于是webpack和vue，所以需要以下命令：</p>
-<pre><code>1. npm install #没有npm的，mac用brew安装。brew install npm。
-2. npm run build #执行完build命令后，会生成dist文件夹，这个文件夹一定要和nginx配置文件的40行所指的目录相同。
+<pre><code>API_BASE = http://localhost:12345
+DEV_HOST = localhost
</code></pre>
-<p>保存以下内容到dolphinscheduler.conf文件：</p>
-<pre><code>
-#user nobody;
-worker_processes 1;
-
-error_log /usr/local/etc/nginx/logs/error.log;
-error_log /usr/local/etc/nginx/logs/error.log notice;
-error_log /usr/local/etc/nginx/logs/error.log info;
-
-pid /usr/local/etc/nginx/logs/nginx.pid;
-
-
-events {
- worker_connections 1024;
-}
-
-
-http {
- include mime.types;
- default_type application/octet-stream;
-
- #log_format main '$remote_addr - $remote_user [$time_local] "$request" '
- # '$status $body_bytes_sent "$http_referer" '
- # '"$http_user_agent" "$http_x_forwarded_for"';
-
- access_log /usr/local/etc/nginx/logs/access.log;
-
- sendfile on;
- #tcp_nopush on;
-
- #keepalive_timeout 0;
- keepalive_timeout 65;
-
- #gzip on;
- server {
- listen 8888;# access port
- server_name localhost;
- #charset koi8-r;
- access_log /usr/local/etc/nginx/logs/host.access.log;
- location / {
- root /xxx/xxx/incubator-dolphinscheduler/dolphinscheduler-ui/dist; #这里为ui目录，需要修改
- index index.html index.html;
- }
- location /dolphinscheduler {
- proxy_pass http://localhost:12345; # interface address
- proxy_set_header Host $host;
- proxy_set_header X-Real-IP $remote_addr;
- proxy_set_header x_real_ipP $remote_addr;
- proxy_set_header remote_addr $remote_addr;
- proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
- proxy_http_version 1.1;
- proxy_connect_timeout 300s;
- proxy_read_timeout 300s;
- proxy_send_timeout 300s;
- proxy_set_header Upgrade $http_upgrade;
- proxy_set_header Connection upgrade;
- }
- #error_page 404 /404.html;
- # redirect server error pages to the static page /50x.html
- #
- error_page 500 502 503 504 /50x.html;
- location = /50x.html {
- root /usr/share/nginx/html;
- }
- }
- include servers/*;
-}
-
-</code></pre>
-<p>修改上面root指向，然后将文件放到nginx配置同级目录,使用以下命令，启动nginx。</p>
-<pre><code>./nginx -c dolphinscheduler.conf
-</code></pre>
-<p>启动ApiApplicationServer类：
-org.apache.dolphinscheduler.api.ApiApplicationServer类main函数增加如下代码：</p>
-<pre><code>System.setProperty("spring.profiles.active", "api");
-</code></pre>
-<p>修改api模块resources目录下apiserver_logback.xml文件，增加以下代码：</p>
-<pre><code><root level="INFO">
- <appender-ref ref="APISERVERLOGFILE" />
- <!-- 增加日志到控制台-->
- <appender-ref ref="STDOUT"/>
-</root>
-</code></pre>
-<pre><code>执行ApiApplicationServer, 然后访问localhost:8888，账号admin/dolphinscheduler123。
+<h4>启动项目</h4>
+<ol>
+<li>
+<p>启动zookeeper<br>
+./bin/zkServer.sh start</p>
+</li>
+<li>
+<p>启动MasterServer，执行org.apache.dolphinscheduler.server.master.MasterServer的main方法,需要设置VM Options:</p>
+<pre><code> -Dlogging.config=classpath:logback-master.xml -Ddruid.mysql.usePingMethod=false
</code></pre>
-<h4>资源中心</h4>
-<p>ds对于资源的存储在本地文件系统/HDFS/S3/minio中，HDFS的mac安装请参考:<a href="https://www.jianshu.com/p/935b4c5e4c25">https://www.jianshu.com/p/935b4c5e4c25</a>
-hdfs下载:<a href="https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.8.5/hadoop-2.8.5.tar.gz">https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.8.5/hadoop-2.8.5.tar.gz</a>
-安装好hdfs后，修改common.properties配置：</p>
-<pre><code>res.upload.startup.type=HDFS
+</li>
+<li>
+<p>启动WorkerServer，执行org.apache.dolphinscheduler.server.worker.WorkerServer的main方法,需要设置VM Options:</p>
+<pre><code> -Dlogging.config=classpath:logback-worker.xml -Ddruid.mysql.usePingMethod=false
</code></pre>
-<p>修改hadoop.properties：</p>
-<pre><code>fs.defaultFS=hdfs://localhost:9000
+</li>
+<li>
+<p>启动ApiApplicationServer，执行org.apache.dolphinscheduler.api.ApiApplicationServer的main方法,需要设置VM Options:</p>
+<pre><code> -Dlogging.config=classpath:logback-api.xml -Dspring.profiles.active=api
</code></pre>
-<p>然后重启ApiApplicationServer即可。</p>
-<p>*<code>注意</code>：上传的文件大小，超过1M下，需要添加nginx参数</p>
-<pre><code>client_max_body_size 100m;
+</li>
+<li>
+<p>,这里暂时不启动其它模块，如果启动其它模块，那么去查询script/dolphinscheduler-daemon.sh文件,设置相应的VM Options</p>
+<pre><code> if [ "$command" = "api-server" ]; then
+ LOG_FILE="-Dlogging.config=classpath:logback-api.xml -Dspring.profiles.active=api"
+ CLASS=org.apache.dolphinscheduler.api.ApiApplicationServer
+ elif [ "$command" = "master-server" ]; then
+ LOG_FILE="-Dlogging.config=classpath:logback-master.xml -Ddruid.mysql.usePingMethod=false"
+ CLASS=org.apache.dolphinscheduler.server.master.MasterServer
+ elif [ "$command" = "worker-server" ]; then
+ LOG_FILE="-Dlogging.config=classpath:logback-worker.xml -Ddruid.mysql.usePingMethod=false"
+ CLASS=org.apache.dolphinscheduler.server.worker.WorkerServer
+ elif [ "$command" = "alert-server" ]; then
+ LOG_FILE="-Dlogback.configurationFile=conf/logback-alert.xml"
+ CLASS=org.apache.dolphinscheduler.alert.AlertServer
+ elif [ "$command" = "logger-server" ]; then
+ CLASS=org.apache.dolphinscheduler.server.log.LoggerServer
+ else
+ echo "Error: No command named \`$command' was found."
+ exit 1
+ fi
</code></pre>
+</li>
+<li>
+<p>启动前端ui模块<br>
+cd dolphinscheduler-ui目录，执行npm run start</p>
+</li>
+</ol>
+<h4>访问项目</h4>
+<ol>
+<li>访问http://localhost:8888<br>
+输入管理员账户admin,密码dolphinscheduler123进行登陆</li>
+</ol>
</div></section><footer class="footer-container"><div class="footer-body"><img src="/img/ds_gray.svg"/><div class="cols-container"><div class="col col-12"><h3>Disclaimer</h3><p>Apache DolphinScheduler (incubating) is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by Incubator.
Incubation is required of all newly accepted projects until a further review indicates
that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects.
diff --git a/zh-cn/docs/development/development-environment-setup.json b/zh-cn/docs/development/development-environment-setup.json
index 1a770b0..e9702b8 100644
--- a/zh-cn/docs/development/development-environment-setup.json
+++ b/zh-cn/docs/development/development-environment-setup.json
@@ -1,6 +1,6 @@
{
"filename": "development-environment-setup.md",
- "__html": "<h4>准备工作</h4>\n<ol>\n<li>\n<p>首先从远端仓库fork <a href=\"https://github.com/apache/incubator-dolphinscheduler\">dolphinscheduler</a> 一份代码到自己的仓库中</p>\n</li>\n<li>\n<p>安装好MySQL/PostgreSQL,Zookeeper,Nginx,Hadoop(可选)</p>\n</li>\n<li>\n<p>把自己仓库clone到本地</p>\n<p><code>git clone https://github.com/apache/incubator-dolphinscheduler.git</code></p>\n</li>\n<li>\n<p>git clone项目后，进入目录，执行以下命令。</p>\n</li>\n</ol>\n<pre><code>1. git branch -a #查看分支\n2. git checkout dev #切换到dev分支\n3. git pull # [...]
+ "__html": "<h4>准备工作</h4>\n<ol>\n<li>\n<p>首先从远端仓库fork <a href=\"https://github.com/apache/incubator-dolphinscheduler\">dolphinscheduler</a> 一份代码到自己的仓库中</p>\n</li>\n<li>\n<p>在开发环境中安装好MySQL/PostgreSQL、JDK、MAVEN</p>\n</li>\n<li>\n<p>把自己仓库clone到本地</p>\n<p><code>git clone https://github.com/apache/incubator-dolphinscheduler.git</code></p>\n</li>\n<li>\n<p>git clone项目后，进入项目目录，执行以下命令。</p>\n</li>\n</ol>\n<pre><code>1. git branch -a #查看分支\n2. git checkout dev #切换到dev分支\n3. git pull #同步分支\n4. [...]
"link": "/zh-cn/docs/development/development-environment-setup.html",
"meta": {}
}
\ No newline at end of file