You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2020/06/08 08:42:37 UTC
[GitHub] [incubator-dolphinscheduler-website] lgcareer opened a new pull request #127: update standalone deployment doc
lgcareer opened a new pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127
update standalone deployment doc
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] dailidong commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
dailidong commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r437955398
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -170,19 +170,20 @@ mysql -uroot -p
- 修改 conf/env 目录下的 `dolphinscheduler_env.sh` 环境变量(以相关用到的软件都安装在/opt/soft下为例)
```shell
- export HADOOP_HOME=/opt/soft/hadoop
- export HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
- #export SPARK_HOME1=/opt/soft/spark1
- export SPARK_HOME2=/opt/soft/spark2
- export PYTHON_HOME=/opt/soft/python
- export JAVA_HOME=/opt/soft/java
- export HIVE_HOME=/opt/soft/hive
- export FLINK_HOME=/opt/soft/flink
- export PATH=$HADOOP_HOME/bin:$SPARK_HOME2/bin:$PYTHON_HOME:$JAVA_HOME/bin:$HIVE_HOME/bin:$PATH:$FLINK_HOME/bin:$PATH
+ export HADOOP_HOME=/opt/soft/hadoop
+ export HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
+ #export SPARK_HOME1=/opt/soft/spark1
+ export SPARK_HOME2=/opt/soft/spark2
+ export PYTHON_HOME=/opt/soft/python
+ export JAVA_HOME=/opt/soft/java
+ export HIVE_HOME=/opt/soft/hive
+ export FLINK_HOME=/opt/soft/flink
+ export DATAX_HOME=/opt/soft/datax/bin/datax.py
+ export PATH=$HADOOP_HOME/bin:$SPARK_HOME1/bin:$SPARK_HOME2/bin:$PYTHON_HOME:$JAVA_HOME/bin:$HIVE_HOME/bin:$PATH:$FLINK_HOME/bin:$DATAX_HOME:$PATH
- ```
+ ```
- `注: 这一步非常重要,例如 JAVA_HOME 和 PATH 是必须要配置的，没有用到的可以忽略或者注释掉；如果找不到.dolphinscheduler_env.sh, 请运行 ls -a`
+ `注: 这一步非常重要,例如 JAVA_HOME 和 PATH 是必须要配置的，没有用到的可以忽略或者注释掉；如果找不到dolphinscheduler_env.sh, 请运行 ls -a`
Review comment:
should remove “如果找不到dolphinscheduler_env.sh, 请运行 ls -a ”
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] lgcareer commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
lgcareer commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r438009705
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -210,40 +211,21 @@ mysql -uroot -p
# 数据库密码, 如果有特殊字符，请使用\转义，需要修改为上面设置的{passowrd}具体值
passowrd="xxx"
- #将DS安装到哪个目录，如: /opt/soft/dolphinscheduler，不同于现在的目录
- installPath="/opt/soft/dolphinscheduler"
-
- #使用哪个用户部署，使用1.3小节创建的用户
- deployUser="dolphinscheduler"
-
#Zookeeper地址，单机本机是localhost:2181，记得把2181端口带上
zkQuorum="localhost:2181"
Review comment:
I have modified.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] dailidong merged pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
dailidong merged pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] lgcareer commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
lgcareer commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r438007899
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -13,7 +13,7 @@
# 2、下载二进制tar.gz包
-- 请下载最新版本的后端安装包至服务器部署目录,比如创建 /opt/dolphinscheduler 做为安装部署目录，下载地址： [下载](https://dolphinscheduler.apache.org/zh-cn/docs/release/download.html) (以1.2.0版本为例)，下载后上传tar包到该目录中，并进行解压
+- 请下载最新版本的后端安装包至服务器部署目录,比如创建 /opt/dolphinscheduler 做为安装部署目录，下载地址： [下载](https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler/1.3.0/https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler/1.3.0/apache-dolphinscheduler-incubating-1.3.0-dolphinscheduler-bin.tar.gz)，下载后上传tar包到该目录中，并进行解压
Review comment:
Thanks,I have modified the download url.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] dailidong commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
dailidong commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r437968707
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -261,35 +243,53 @@ mysql -uroot -p
# TLS协议的邮箱设置为true，否则设置为false
starttlsEnable="true"
- # 邮件服务地址值，参考上面 mailServerHost
- sslTrust="smtp.exmail.qq.com"
-
# 开启SSL协议的邮箱配置为true，否则为false。注意: starttlsEnable和sslEnable不能同时为true
sslEnable="false"
- # excel下载路径
- xlsFilePath="/tmp/xls"
+ # 邮件服务地址值，参考上面 mailServerHost
+ sslTrust="smtp.qq.com"
+
# 业务用到的比如sql等资源文件上传到哪里，可以设置：HDFS,S3,NONE，单机如果想使用本地文件系统，请配置为HDFS，因为HDFS支持本地文件系统；如果不需要资源上传功能请选择NONE。强调一点：使用本地文件系统不需要部署hadoop
resourceStorageType="HDFS"
- # 资源上传根路径,主持HDFS和S3,由于hdfs支持本地文件系统，需要确保本地文件夹存在且有读写权限
- resourceUploadPath="/data/dolphinscheduler"
-
- # 具备权限创建resourceUploadPath的用户
- hdfsRootUser="hdfs"
-
# 这里以保存到本地文件系统为例
- #注：但是如果你想上传到HDFS的话，NameNode启用了HA，则需要将core-site.xml和hdfs-site.xml放到conf目录下，本例即是放到/opt/dolphinscheduler/conf下面，并配置namenode cluster名称；如果NameNode不是HA,则修改为具体的ip或者主机名即可
+ #注：但是如果你想上传到HDFS的话，NameNode启用了HA，则需要将hadoop的配置文件core-site.xml和hdfs-site.xml放到conf目录下，本例即是放到/opt/dolphinscheduler/conf下面，并配置namenode cluster名称；如果NameNode不是HA,则修改为具体的ip或者主机名即可
defaultFS="file:///data/dolphinscheduler" #hdfs://{具体的ip/主机名}:8020
Review comment:
not right
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] dailidong commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
dailidong commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r437901779
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -13,7 +13,7 @@
# 2、下载二进制tar.gz包
-- 请下载最新版本的后端安装包至服务器部署目录,比如创建 /opt/dolphinscheduler 做为安装部署目录，下载地址： [下载](https://dolphinscheduler.apache.org/zh-cn/docs/release/download.html) (以1.2.0版本为例)，下载后上传tar包到该目录中，并进行解压
+- 请下载最新版本的后端安装包至服务器部署目录,比如创建 /opt/dolphinscheduler 做为安装部署目录，下载地址： [下载](https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler/1.3.0/https://dist.apache.org/repos/dist/dev/incubator/dolphinscheduler/1.3.0/apache-dolphinscheduler-incubating-1.3.0-dolphinscheduler-bin.tar.gz)，下载后上传tar包到该目录中，并进行解压
Review comment:
url address not right , it will 404 when you visit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] dailidong commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
dailidong commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r437966068
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -210,40 +211,21 @@ mysql -uroot -p
# 数据库密码, 如果有特殊字符，请使用\转义，需要修改为上面设置的{passowrd}具体值
passowrd="xxx"
- #将DS安装到哪个目录，如: /opt/soft/dolphinscheduler，不同于现在的目录
- installPath="/opt/soft/dolphinscheduler"
-
- #使用哪个用户部署，使用1.3小节创建的用户
- deployUser="dolphinscheduler"
-
#Zookeeper地址，单机本机是localhost:2181，记得把2181端口带上
zkQuorum="localhost:2181"
Review comment:
cluster is "192.168.xx.xx:2181,192.168.xx.xx:2181,192.168.xx.xx:2181"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] lgcareer commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
lgcareer commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r438011655
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -170,19 +170,20 @@ mysql -uroot -p
- 修改 conf/env 目录下的 `dolphinscheduler_env.sh` 环境变量(以相关用到的软件都安装在/opt/soft下为例)
```shell
- export HADOOP_HOME=/opt/soft/hadoop
- export HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
- #export SPARK_HOME1=/opt/soft/spark1
- export SPARK_HOME2=/opt/soft/spark2
- export PYTHON_HOME=/opt/soft/python
- export JAVA_HOME=/opt/soft/java
- export HIVE_HOME=/opt/soft/hive
- export FLINK_HOME=/opt/soft/flink
- export PATH=$HADOOP_HOME/bin:$SPARK_HOME2/bin:$PYTHON_HOME:$JAVA_HOME/bin:$HIVE_HOME/bin:$PATH:$FLINK_HOME/bin:$PATH
+ export HADOOP_HOME=/opt/soft/hadoop
+ export HADOOP_CONF_DIR=/opt/soft/hadoop/etc/hadoop
+ #export SPARK_HOME1=/opt/soft/spark1
+ export SPARK_HOME2=/opt/soft/spark2
+ export PYTHON_HOME=/opt/soft/python
+ export JAVA_HOME=/opt/soft/java
+ export HIVE_HOME=/opt/soft/hive
+ export FLINK_HOME=/opt/soft/flink
+ export DATAX_HOME=/opt/soft/datax/bin/datax.py
+ export PATH=$HADOOP_HOME/bin:$SPARK_HOME1/bin:$SPARK_HOME2/bin:$PYTHON_HOME:$JAVA_HOME/bin:$HIVE_HOME/bin:$PATH:$FLINK_HOME/bin:$DATAX_HOME:$PATH
- ```
+ ```
- `注: 这一步非常重要,例如 JAVA_HOME 和 PATH 是必须要配置的，没有用到的可以忽略或者注释掉；如果找不到.dolphinscheduler_env.sh, 请运行 ls -a`
+ `注: 这一步非常重要,例如 JAVA_HOME 和 PATH 是必须要配置的，没有用到的可以忽略或者注释掉；如果找不到dolphinscheduler_env.sh, 请运行 ls -a`
Review comment:
Removed done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] dailidong commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
dailidong commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r437970062
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -261,35 +243,53 @@ mysql -uroot -p
# TLS协议的邮箱设置为true，否则设置为false
starttlsEnable="true"
- # 邮件服务地址值，参考上面 mailServerHost
- sslTrust="smtp.exmail.qq.com"
-
# 开启SSL协议的邮箱配置为true，否则为false。注意: starttlsEnable和sslEnable不能同时为true
sslEnable="false"
- # excel下载路径
- xlsFilePath="/tmp/xls"
+ # 邮件服务地址值，参考上面 mailServerHost
+ sslTrust="smtp.qq.com"
+
# 业务用到的比如sql等资源文件上传到哪里，可以设置：HDFS,S3,NONE，单机如果想使用本地文件系统，请配置为HDFS，因为HDFS支持本地文件系统；如果不需要资源上传功能请选择NONE。强调一点：使用本地文件系统不需要部署hadoop
resourceStorageType="HDFS"
- # 资源上传根路径,主持HDFS和S3,由于hdfs支持本地文件系统，需要确保本地文件夹存在且有读写权限
- resourceUploadPath="/data/dolphinscheduler"
-
- # 具备权限创建resourceUploadPath的用户
- hdfsRootUser="hdfs"
-
# 这里以保存到本地文件系统为例
- #注：但是如果你想上传到HDFS的话，NameNode启用了HA，则需要将core-site.xml和hdfs-site.xml放到conf目录下，本例即是放到/opt/dolphinscheduler/conf下面，并配置namenode cluster名称；如果NameNode不是HA,则修改为具体的ip或者主机名即可
+ #注：但是如果你想上传到HDFS的话，NameNode启用了HA，则需要将hadoop的配置文件core-site.xml和hdfs-site.xml放到conf目录下，本例即是放到/opt/dolphinscheduler/conf下面，并配置namenode cluster名称；如果NameNode不是HA,则修改为具体的ip或者主机名即可
defaultFS="file:///data/dolphinscheduler" #hdfs://{具体的ip/主机名}:8020
-
# 如果ResourceManager是HA，则配置为ResourceManager节点的主备ip或者hostname,比如"192.168.xx.xx,192.168.xx.xx"，否则如果是单ResourceManager或者根本没用到yarn,请配置yarnHaIps=""即可，我这里没用到yarn，配置为""
yarnHaIps=""
# 如果是单ResourceManager，则配置为ResourceManager节点ip或主机名，否则保持默认值即可。我这里没用到yarn，保持默认
singleYarnIp="ark1"
Review comment:
# if not use hadoop resourcemanager, please keep default value; if resourcemanager HA enable, please type the HA ips ; if resourcemanager is single, make this value empty
yarnHaIps="192.168.xx.xx,192.168.xx.xx"
# if resourcemanager HA enable or not use resourcemanager, please skip this value setting; If resourcemanager is single, you only need to replace yarnIp1 to actual resourcemanager hostname.
singleYarnIp="yarnIp1"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-dolphinscheduler-website] dailidong commented on a change in pull request #127: update standalone deployment doc
Posted by GitBox <gi...@apache.org>.
dailidong commented on a change in pull request #127:
URL: https://github.com/apache/incubator-dolphinscheduler-website/pull/127#discussion_r437970062
##########
File path: docs/zh-cn/1.3.0/user_doc/cluster-deployment.md
##########
@@ -261,35 +243,53 @@ mysql -uroot -p
# TLS协议的邮箱设置为true，否则设置为false
starttlsEnable="true"
- # 邮件服务地址值，参考上面 mailServerHost
- sslTrust="smtp.exmail.qq.com"
-
# 开启SSL协议的邮箱配置为true，否则为false。注意: starttlsEnable和sslEnable不能同时为true
sslEnable="false"
- # excel下载路径
- xlsFilePath="/tmp/xls"
+ # 邮件服务地址值，参考上面 mailServerHost
+ sslTrust="smtp.qq.com"
+
# 业务用到的比如sql等资源文件上传到哪里，可以设置：HDFS,S3,NONE，单机如果想使用本地文件系统，请配置为HDFS，因为HDFS支持本地文件系统；如果不需要资源上传功能请选择NONE。强调一点：使用本地文件系统不需要部署hadoop
resourceStorageType="HDFS"
- # 资源上传根路径,主持HDFS和S3,由于hdfs支持本地文件系统，需要确保本地文件夹存在且有读写权限
- resourceUploadPath="/data/dolphinscheduler"
-
- # 具备权限创建resourceUploadPath的用户
- hdfsRootUser="hdfs"
-
# 这里以保存到本地文件系统为例
- #注：但是如果你想上传到HDFS的话，NameNode启用了HA，则需要将core-site.xml和hdfs-site.xml放到conf目录下，本例即是放到/opt/dolphinscheduler/conf下面，并配置namenode cluster名称；如果NameNode不是HA,则修改为具体的ip或者主机名即可
+ #注：但是如果你想上传到HDFS的话，NameNode启用了HA，则需要将hadoop的配置文件core-site.xml和hdfs-site.xml放到conf目录下，本例即是放到/opt/dolphinscheduler/conf下面，并配置namenode cluster名称；如果NameNode不是HA,则修改为具体的ip或者主机名即可
defaultFS="file:///data/dolphinscheduler" #hdfs://{具体的ip/主机名}:8020
-
# 如果ResourceManager是HA，则配置为ResourceManager节点的主备ip或者hostname,比如"192.168.xx.xx,192.168.xx.xx"，否则如果是单ResourceManager或者根本没用到yarn,请配置yarnHaIps=""即可，我这里没用到yarn，配置为""
yarnHaIps=""
# 如果是单ResourceManager，则配置为ResourceManager节点ip或主机名，否则保持默认值即可。我这里没用到yarn，保持默认
singleYarnIp="ark1"
Review comment:
if not use hadoop resourcemanager, please keep default value; if resourcemanager HA enable, please type the HA ips ; if resourcemanager is single, make this value empty
yarnHaIps="192.168.xx.xx,192.168.xx.xx"
if resourcemanager HA enable or not use resourcemanager, please skip this value setting; If resourcemanager is single, you only need to replace yarnIp1 to actual resourcemanager hostname.
singleYarnIp="yarnIp1"
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org