You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by GitBox <gi...@apache.org> on 2022/05/27 06:20:16 UTC
[GitHub] [dolphinscheduler] GavinGYM opened a new pull request, #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
GavinGYM opened a new pull request, #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264
## Purpose of the pull request
This pull request added Local File Resource Configuration Guide to the document, updated file path for common.properties, and removed unnecessary tips.
## Brief change log
*Modified docs/en/guide/resource/configuration.md*
*Modified docs/zh/guide/resource/configuration.md*
## Verify this pull request
This pull request is code cleanup without any test coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r884401986
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
Review Comment:
```suggestion
For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
```
##########
docs/docs/zh/guide/resource/configuration.md:
##########
@@ -2,6 +2,16 @@
资源中心通常用于上传文件、 UDF 函数，以及任务组管理等操作。针对单机环境可以选择本地文件目录作为上传文件夹（此操作不需要部署 Hadoop）。当然也可以选择上传到 Hadoop or MinIO 集群上，此时则需要有 Hadoop（2.6+）或者 MinIOn 等相关环境。
+## 本地资源配置
+在单机环境下，可以选择使用本地文件目录作为上传文件夹（无需部署Hadoop），此时需要进行如下配置：
+
+### 配置 common.properties 文件
Review Comment:
```suggestion
### 配置 `common.properties` 文件
```
##########
docs/docs/zh/guide/resource/configuration.md:
##########
@@ -2,6 +2,16 @@
资源中心通常用于上传文件、 UDF 函数，以及任务组管理等操作。针对单机环境可以选择本地文件目录作为上传文件夹（此操作不需要部署 Hadoop）。当然也可以选择上传到 Hadoop or MinIO 集群上，此时则需要有 Hadoop（2.6+）或者 MinIOn 等相关环境。
+## 本地资源配置
+在单机环境下，可以选择使用本地文件目录作为上传文件夹（无需部署Hadoop），此时需要进行如下配置：
Review Comment:
```suggestion
在单机环境下，可以选择使用本地文件目录作为上传文件夹（无需部署Hadoop），此时需要进行如下配置：
```
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the `common.properties`
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change `data.basedir.path` to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions, such as: `data.basedir.path=/tmp/dolphinscheduler`.
Review Comment:
```suggestion
- Change `data.basedir.path` to the local directory path. Please make sure the user who deploy dolphinscheduler have read and write permissions, such as: `data.basedir.path=/tmp/dolphinscheduler`. And the directory you configured will be auto-created if it does not exists.
```
##########
docs/docs/zh/guide/resource/configuration.md:
##########
@@ -2,6 +2,16 @@
资源中心通常用于上传文件、 UDF 函数，以及任务组管理等操作。针对单机环境可以选择本地文件目录作为上传文件夹（此操作不需要部署 Hadoop）。当然也可以选择上传到 Hadoop or MinIO 集群上，此时则需要有 Hadoop（2.6+）或者 MinIOn 等相关环境。
+## 本地资源配置
+在单机环境下，可以选择使用本地文件目录作为上传文件夹（无需部署Hadoop），此时需要进行如下配置：
+
+### 配置 common.properties 文件
+
+对以下路径的文件进行配置：`api-server/conf/common.properties` 和 `worker-server/conf/common.properties`
+
+- 将 `data.basedir.path` 改为本地存储路径，请确保路径存在，并且部署 DolphinScheduler 的用户拥有读写权限，例如：`data.basedir.path=/tmp/dolphinscheduler`
Review Comment:
```suggestion
- 将 `data.basedir.path` 改为本地存储路径，请确保部署 DolphinScheduler 的用户拥有读写权限，例如：`data.basedir.path=/tmp/dolphinscheduler`。当路径不存在时会自动创建文件夹
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883346274
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
+- Modify two fields, `resource.storage.type=HDFS` and `fs.defaultFS=file:///`.
Review Comment:
Is it better to use ```Modify the following two parameters```?
##########
docs/docs/zh/guide/resource/configuration.md:
##########
@@ -2,6 +2,16 @@
资源中心通常用于上传文件、 UDF 函数，以及任务组管理等操作。针对单机环境可以选择本地文件目录作为上传文件夹（此操作不需要部署 Hadoop）。当然也可以选择上传到 Hadoop or MinIO 集群上，此时则需要有 Hadoop（2.6+）或者 MinIOn 等相关环境。
+## 本地资源配置
+在单机环境下，可以选择使用本地文件目录作为上传文件夹（无需部署Hadoop），此时需要进行如下配置：
+
+### 配置 common.properties 文件
+
+对以下路径的文件进行配置：`api-server/conf/common.properties` 和 `worker-server/conf/common.properties`
+
+- 将`data.basedir.path`改为本地存储路径，请确保路径存在并拥有读写权限，例如：`data.basedir.path=C:/tmp/dolphinscheduler`
Review Comment:
Same as english's doc.
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
Review Comment:
It's recommanded use it to describe ``Change `data.basedir.path` to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions. For example `data.basedir.path=/tmp/dolphinscheduler` ``.
##########
docs/docs/zh/guide/resource/configuration.md:
##########
@@ -2,6 +2,16 @@
资源中心通常用于上传文件、 UDF 函数，以及任务组管理等操作。针对单机环境可以选择本地文件目录作为上传文件夹（此操作不需要部署 Hadoop）。当然也可以选择上传到 Hadoop or MinIO 集群上，此时则需要有 Hadoop（2.6+）或者 MinIOn 等相关环境。
+## 本地资源配置
+在单机环境下，可以选择使用本地文件目录作为上传文件夹（无需部署Hadoop），此时需要进行如下配置：
+
+### 配置 common.properties 文件
+
+对以下路径的文件进行配置：`api-server/conf/common.properties` 和 `worker-server/conf/common.properties`
+
+- 将`data.basedir.path`改为本地存储路径，请确保路径存在并拥有读写权限，例如：`data.basedir.path=C:/tmp/dolphinscheduler`
+- 修改两个字段，分别是`resource.storage.type=HDFS`和`fs.defaultFS=file:///`。
Review Comment:
Same as english's doc.
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
+- Modify two fields, `resource.storage.type=HDFS` and `fs.defaultFS=file:///`.
+
## HDFS Resource Configuration
When it is necessary to use the Resource Center to create or upload relevant files, all files and resources will be stored on HDFS. Therefore the following configuration is required.
### Configuring the common.properties
-After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths The following paths need to be configured: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
+After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
Review Comment:
I think we should keep it as it is which is more accurate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#issuecomment-1140665285
> Hi @GavinGYM could we combine this PR to #10265, I think is better to submit PR together with code and docs instead of separating them WDYT @SbloodyS @QuakeWang
I got it, the PR author is not the same, pls ignore this comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883349371
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
Review Comment:
BTW, currently we only support Linux type system in ds. Running ds in windows may cause unexpected problems. So we'd better not use some content with windows features in the documents.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r884402811
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions, such as: `data.basedir.path=/tmp/dolphinscheduler`.
Review Comment:
cc @GavinGYM
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions, such as: `data.basedir.path=/tmp/dolphinscheduler`.
Review Comment:
FYI @GavinGYM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] GavinGYM commented on pull request #10264: [docs] Added local file configuration guide for resource center
Posted by GitBox <gi...@apache.org>.
GavinGYM commented on PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#issuecomment-1140672749
> BTW, capitalizing the first word of a sentence is enough for both PR or issue title @GavinGYM
Got it. I'll pay attention next time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie merged pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
zhongjiajie merged PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] GavinGYM commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
GavinGYM commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883355051
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
+- Modify two fields, `resource.storage.type=HDFS` and `fs.defaultFS=file:///`.
+
## HDFS Resource Configuration
When it is necessary to use the Resource Center to create or upload relevant files, all files and resources will be stored on HDFS. Therefore the following configuration is required.
### Configuring the common.properties
-After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths The following paths need to be configured: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
+After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
Review Comment:
> I think we should keep it as it is which is more accurate.
The text repeats itself here. `you will need to configure the following paths` and `The following paths need to be configured`. I deleted one of them and kept the other.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883381801
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
Review Comment:
> It's recommanded use it to describe `` Change `data.basedir.path` to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions. For example `data.basedir.path=/tmp/dolphinscheduler` ``.
I think we should specify ```the user who deploy dolphinscheduler have read and write permissions```. Otherwise it may cause misunderstand.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] GavinGYM commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
GavinGYM commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883384230
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
Review Comment:
> > It's recommanded use it to describe `` Change `data.basedir.path` to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions. For example `data.basedir.path=/tmp/dolphinscheduler` ``.
>
> I think we should specify `the user who deploy dolphinscheduler have read and write permissions`. Otherwise it may cause misunderstanding.
Okay, I will specify it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] GavinGYM commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
GavinGYM commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883354675
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
+- Modify two fields, `resource.storage.type=HDFS` and `fs.defaultFS=file:///`.
+
## HDFS Resource Configuration
When it is necessary to use the Resource Center to create or upload relevant files, all files and resources will be stored on HDFS. Therefore the following configuration is required.
### Configuring the common.properties
-After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths The following paths need to be configured: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
+After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
Review Comment:
The text repeats itself here. `you will need to configure the following paths` and `The following paths need to be configured`. I deleted one of them and kept the other.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r884398846
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions, such as: `data.basedir.path=/tmp/dolphinscheduler`.
Review Comment:
Same as the previous comment
```suggestion
- Change `data.basedir.path` to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions, such as: `data.basedir.path=/tmp/dolphinscheduler`.
```
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
Review Comment:
If we using code or configuration file in the docs, should better using the code block in markdown syntax
```suggestion
### Configuring the `common.properties`
```
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions, such as: `data.basedir.path=/tmp/dolphinscheduler`.
Review Comment:
I think it will auto-create directory if not exists, so maybe we could delete the `Please make sure the directory exists` https://github.com/apache/dolphinscheduler/blob/996790ce9ea48bd96aed6687d31afe931fc70893/dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/HadoopUtils.java#L103-L112
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883381801
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
Review Comment:
> It's recommanded use it to describe `` Change `data.basedir.path` to the local directory path. Please make sure the directory exists and the user who deploy dolphinscheduler have read and write permissions. For example `data.basedir.path=/tmp/dolphinscheduler` ``.
I think we should specify ```the user who deploy dolphinscheduler have read and write permissions```. Otherwise it may cause misunderstanding.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] SbloodyS commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
SbloodyS commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883359779
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
+- Modify two fields, `resource.storage.type=HDFS` and `fs.defaultFS=file:///`.
+
## HDFS Resource Configuration
When it is necessary to use the Resource Center to create or upload relevant files, all files and resources will be stored on HDFS. Therefore the following configuration is required.
### Configuring the common.properties
-After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths The following paths need to be configured: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
+After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
Review Comment:
My mistake. Should we keep ```The following paths need to be configured``` instead of ```you will need to configure the following paths```? @GavinGYM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] GavinGYM commented on pull request #10264: [docs] Added local file configuration guide for resource center
Posted by GitBox <gi...@apache.org>.
GavinGYM commented on PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#issuecomment-1140671745
> LGTM, thank @GavinGYM
My pleasure.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#issuecomment-1140660875
Hi @GavinGYM could we combine this PR to #10265, I think is better to submit PR together with code and docs instead of separating them WDYT @SbloodyS @QuakeWang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] zhongjiajie commented on pull request #10264: [docs] Added local file configuration guide for resource center
Posted by GitBox <gi...@apache.org>.
zhongjiajie commented on PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#issuecomment-1140671138
BTW, capitalizing the first word of a sentence is enough for both PR or issue title @GavinGYM
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] QuakeWang commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
QuakeWang commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r884265954
##########
docs/docs/zh/guide/resource/configuration.md:
##########
@@ -2,6 +2,16 @@
资源中心通常用于上传文件、 UDF 函数，以及任务组管理等操作。针对单机环境可以选择本地文件目录作为上传文件夹（此操作不需要部署 Hadoop）。当然也可以选择上传到 Hadoop or MinIO 集群上，此时则需要有 Hadoop（2.6+）或者 MinIOn 等相关环境。
+## 本地资源配置
+在单机环境下，可以选择使用本地文件目录作为上传文件夹（无需部署Hadoop），此时需要进行如下配置：
+
+### 配置 common.properties 文件
+
+对以下路径的文件进行配置：`api-server/conf/common.properties` 和 `worker-server/conf/common.properties`
+
+- 将`data.basedir.path`改为本地存储路径，请确保路径存在，并且部署DolphinScheduler的用户拥有读写权限，例如：`data.basedir.path=/tmp/dolphinscheduler`
Review Comment:
```suggestion
- 将 `data.basedir.path` 改为本地存储路径，请确保路径存在，并且部署 DolphinScheduler 的用户拥有读写权限，例如：`data.basedir.path=/tmp/dolphinscheduler`
```
ref :[document specification](https://dolphinscheduler.apache.org/en-us/community/development/document.html).
Add a space between Chinese and English, for the purpose of document specification.
##########
docs/docs/zh/guide/resource/configuration.md:
##########
@@ -2,6 +2,16 @@
资源中心通常用于上传文件、 UDF 函数，以及任务组管理等操作。针对单机环境可以选择本地文件目录作为上传文件夹（此操作不需要部署 Hadoop）。当然也可以选择上传到 Hadoop or MinIO 集群上，此时则需要有 Hadoop（2.6+）或者 MinIOn 等相关环境。
+## 本地资源配置
+在单机环境下，可以选择使用本地文件目录作为上传文件夹（无需部署Hadoop），此时需要进行如下配置：
+
+### 配置 common.properties 文件
+
+对以下路径的文件进行配置：`api-server/conf/common.properties` 和 `worker-server/conf/common.properties`
+
+- 将`data.basedir.path`改为本地存储路径，请确保路径存在，并且部署DolphinScheduler的用户拥有读写权限，例如：`data.basedir.path=/tmp/dolphinscheduler`
+- 修改下列两个参数，分别是`resource.storage.type=HDFS`和`fs.defaultFS=file:///`。
Review Comment:
```suggestion
- 修改下列两个参数，分别是 `resource.storage.type=HDFS` 和 `fs.defaultFS=file:///`。
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [dolphinscheduler] GavinGYM commented on a diff in pull request #10264: [Feature-9385][docs] Added Local File Resource Configuration Guide to the document
Posted by GitBox <gi...@apache.org>.
GavinGYM commented on code in PR #10264:
URL: https://github.com/apache/dolphinscheduler/pull/10264#discussion_r883362059
##########
docs/docs/en/guide/resource/configuration.md:
##########
@@ -2,13 +2,23 @@
The Resource Center is usually used for operations such as uploading files, UDF functions, and task group management. You can appoint the local file directory as the upload directory for a single machine (this operation does not need to deploy Hadoop). Or you can also upload to a Hadoop or MinIO cluster, at this time, you need to have Hadoop (2.6+) or MinIO or other related environments.
+## Local File Resource Configuration
+For a single machine, you can choose to use local file directory as the upload directory (no need to deploy Hadoop) by making the following configuration.
+
+### Configuring the common.properties
+
+Configure the file in the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`.
+
+- Change 'data.basedir.path' to the local directory path. Please make sure the directory exists and have read write permissions, such as: `data.basedir.path=C:/tmp/dolphinscheduler`.
+- Modify two fields, `resource.storage.type=HDFS` and `fs.defaultFS=file:///`.
+
## HDFS Resource Configuration
When it is necessary to use the Resource Center to create or upload relevant files, all files and resources will be stored on HDFS. Therefore the following configuration is required.
### Configuring the common.properties
-After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths The following paths need to be configured: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
+After version 3.0.0-alpha, if you want to upload resources using HDFS or S3 from the Resource Center, you will need to configure the following paths: `api-server/conf/common.properties` and `worker-server/conf/common.properties`. This can be found as follows.
Review Comment:
> My mistake. Should we keep `The following paths need to be configured` instead of `you will need to configure the following paths`? @GavinGYM
Sure!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@dolphinscheduler.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org