You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@dolphinscheduler.apache.org by zh...@apache.org on 2022/05/24 15:12:13 UTC
[dolphinscheduler] 13/16: [doc] Update kubernetes' FAQ (#10221)
This is an automated email from the ASF dual-hosted git repository.
zhongjiajie pushed a commit to branch 3.0.0-beta-prepare
in repository https://gitbox.apache.org/repos/asf/dolphinscheduler.git
commit 00e11622865077391c01026540e3e894bf4faed0
Author: 旺阳 <qi...@cisco.com>
AuthorDate: Tue May 24 17:00:57 2022 +0800
[doc] Update kubernetes' FAQ (#10221)
(cherry picked from commit c12848a4ff7fa2fd307c0c18acef40c410d4db99)
---
docs/docs/en/guide/installation/kubernetes.md | 89 ++++++++++++--------------
docs/docs/zh/guide/installation/kubernetes.md | 91 ++++++++++++---------------
2 files changed, 79 insertions(+), 101 deletions(-)
diff --git a/docs/docs/en/guide/installation/kubernetes.md b/docs/docs/en/guide/installation/kubernetes.md
index 572bf19e7b..4162b03217 100644
--- a/docs/docs/en/guide/installation/kubernetes.md
+++ b/docs/docs/en/guide/installation/kubernetes.md
@@ -186,24 +186,29 @@ kubectl scale --replicas=6 sts dolphinscheduler-worker -n test # with test names
> Because of the commercial license, we cannot directly use the driver of MySQL.
>
-> If you want to use MySQL, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions:
+> If you want to use MySQL, you can build a new image based on the `apache/dolphinscheduler-<service>` image follow the following instructions:
+>
+> Since version 3.0.0, dolphinscheduler has been microserviced and the change of metadata storage requires replacing all services with MySQL driver, which including dolphinscheduler-tools, dolphinscheduler-master, dolphinscheduler-worker, dolphinscheduler-api, dolphinscheduler-alert-server
1. Download the MySQL driver [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar).
2. Create a new `Dockerfile` to add MySQL driver:
```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
+FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:3.0.0-beta-1
+# For example
+# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-tools:3.0.0-beta-1
+
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
```
3. Build a new docker image including MySQL driver:
```
-docker build -t apache/dolphinscheduler:mysql-driver .
+docker build -t apache/dolphinscheduler-<service>:mysql-driver .
```
-4. Push the docker image `apache/dolphinscheduler:mysql-driver` to a docker registry.
+4. Push the docker image `apache/dolphinscheduler-<service>:mysql-driver` to a docker registry.
5. Modify image `repository` and update `tag` to `mysql-driver` in `values.yaml`.
@@ -214,7 +219,6 @@ docker build -t apache/dolphinscheduler:mysql-driver .
```yaml
externalDatabase:
type: "mysql"
- driver: "com.mysql.jdbc.Driver"
host: "localhost"
port: "3306"
username: "root"
@@ -225,70 +229,53 @@ externalDatabase:
8. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**).
-### How to Support MySQL Datasource in `Datasource manage`?
+### How to Support MySQL or Oracle Datasource in `Datasource manage`?
-> Because of the commercial license, we cannot directly use the driver of MySQL.
+> Because of the commercial license, we cannot directly use the driver of MySQL or Oracle.
>
-> If you want to add MySQL datasource, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions:
+> If you want to add MySQL or Oracle datasource, you can build a new image based on the `apache/dolphinscheduler-<service>` image follow the following instructions:
+>
+> You need to change the two service images including dolphinscheduler-worker, dolphinscheduler-api.
1. Download the MySQL driver [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar).
+or download the Oracle driver [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (such as `ojdbc8-19.9.0.0.jar`)
-2. Create a new `Dockerfile` to add MySQL driver:
+2. Create a new `Dockerfile` to add MySQL or Oracle driver:
```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
-COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
-```
+FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:3.0.0-beta-1
+# For example
+# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.0.0-beta-1
-3. Build a new docker image including MySQL driver:
-
-```
-docker build -t apache/dolphinscheduler:mysql-driver .
-```
-
-4. Push the docker image `apache/dolphinscheduler:mysql-driver` to a docker registry.
-
-5. Modify image `repository` and update `tag` to `mysql-driver` in `values.yaml`.
-
-6. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**).
-
-7. Add a MySQL datasource in `Datasource manage`.
-
-### How to Support Oracle Datasource in `Datasource manage`?
-
-> Because of the commercial license, we cannot directly use the driver of Oracle.
->
-> If you want to add Oracle datasource, you can build a new image based on the `apache/dolphinscheduler` image follow the following instructions:
-
-1. Download the Oracle driver [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (such as `ojdbc8-19.9.0.0.jar`)
-
-2. Create a new `Dockerfile` to add Oracle driver:
+# If you want to support MySQL Datasource
+COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
-```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
+# If you want to support Oracle Datasource
COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib
```
-3. Build a new docker image including Oracle driver:
+3. Build a new docker image including MySQL or Oracle driver:
```
-docker build -t apache/dolphinscheduler:oracle-driver .
+docker build -t apache/dolphinscheduler-<service>:new-driver .
```
-4. Push the docker image `apache/dolphinscheduler:oracle-driver` to a docker registry.
+4. Push the docker image `apache/dolphinscheduler-<service>:new-driver` to a docker registry.
-5. Modify image `repository` and update `tag` to `oracle-driver` in `values.yaml`.
+5. Modify image `repository` and update `tag` to `new-driver` in `values.yaml`.
6. Run a DolphinScheduler release in Kubernetes (See **Install DolphinScheduler**).
-7. Add an Oracle datasource in `Datasource manage`.
+7. Add a MySQL or Oracle datasource in `Datasource manage`.
### How to Support Python 2 pip and Custom requirements.txt?
+> Just change the image of the dolphinscheduler-worker service.
+
1. Create a new `Dockerfile` to install pip:
```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
+FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.0.0-beta-1
COPY requirements.txt /tmp
RUN apt-get update && \
apt-get install -y --no-install-recommends python-pip && \
@@ -299,16 +286,16 @@ RUN apt-get update && \
The command will install the default **pip 18.1**. If you upgrade the pip, just add the following command.
```
- pip install --no-cache-dir -U pip && \
+pip install --no-cache-dir -U pip && \
```
2. Build a new docker image including pip:
```
-docker build -t apache/dolphinscheduler:pip .
+docker build -t apache/dolphinscheduler-worker:pip .
```
-3. Push the docker image `apache/dolphinscheduler:pip` to a docker registry.
+3. Push the docker image `apache/dolphinscheduler-worker:pip` to a docker registry.
4. Modify image `repository` and update `tag` to `pip` in `values.yaml`.
@@ -318,10 +305,12 @@ docker build -t apache/dolphinscheduler:pip .
### How to Support Python 3?
+> Just change the image of the dolphinscheduler-worker service.
+
1. Create a new `Dockerfile` to install Python 3:
```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
+FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.0.0-beta-1
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 && \
rm -rf /var/lib/apt/lists/*
@@ -330,16 +319,16 @@ RUN apt-get update && \
The command will install the default **Python 3.7.3**. If you also want to install **pip3**, just replace `python3` with `python3-pip` like:
```
- apt-get install -y --no-install-recommends python3-pip && \
+apt-get install -y --no-install-recommends python3-pip && \
```
2. Build a new docker image including Python 3:
```
-docker build -t apache/dolphinscheduler:python3 .
+docker build -t apache/dolphinscheduler-worker:python3 .
```
-3. Push the docker image `apache/dolphinscheduler:python3` to a docker registry.
+3. Push the docker image `apache/dolphinscheduler-worker:python3` to a docker registry.
4. Modify image `repository` and update `tag` to `python3` in `values.yaml`.
diff --git a/docs/docs/zh/guide/installation/kubernetes.md b/docs/docs/zh/guide/installation/kubernetes.md
index 5a97ed2467..569fdb0d00 100644
--- a/docs/docs/zh/guide/installation/kubernetes.md
+++ b/docs/docs/zh/guide/installation/kubernetes.md
@@ -187,24 +187,29 @@ kubectl scale --replicas=6 sts dolphinscheduler-worker -n test # with test names
> 由于商业许可证的原因，我们不能直接使用 MySQL 的驱动包.
>
-> 如果你要使用 MySQL, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建.
+> 如果你要使用 MySQL, 你可以基于官方镜像 `apache/dolphinscheduler-<service>` 进行构建.
+>
+> 从3.0.0版本起，dolphinscheduler已经微服务化，更改元数据存储需要对把所有的服务都替换为 MySQL 驱动包，包括 dolphinscheduler-tools, dolphinscheduler-master, dolphinscheduler-worker, dolphinscheduler-api, dolphinscheduler-alert-server .
1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar)
2. 创建一个新的 `Dockerfile`，用于添加 MySQL 的驱动包:
```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
+FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:3.0.0-beta-1
+# For example
+# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-tools:3.0.0-beta-1
+
COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
```
3. 构建一个包含 MySQL 驱动包的新镜像:
```
-docker build -t apache/dolphinscheduler:mysql-driver .
+docker build -t apache/dolphinscheduler-<service>:mysql-driver .
```
-4. 推送 docker 镜像 `apache/dolphinscheduler:mysql-driver` 到一个 docker registry 中
+4. 推送 docker 镜像 `apache/dolphinscheduler-<service>:mysql-driver` 到一个 docker registry 中
5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `mysql-driver`
@@ -215,7 +220,6 @@ docker build -t apache/dolphinscheduler:mysql-driver .
```yaml
externalDatabase:
type: "mysql"
- driver: "com.mysql.jdbc.Driver"
host: "localhost"
port: "3306"
username: "root"
@@ -226,70 +230,53 @@ externalDatabase:
8. 部署 dolphinscheduler (详见**安装 dolphinscheduler**)
-### 如何在数据源中心支持 MySQL 数据源？
+### 如何在数据源中心支持 MySQL 或者 Oracle 数据源？
-> 由于商业许可证的原因，我们不能直接使用 MySQL 的驱动包.
+> 由于商业许可证的原因，我们不能直接使用 MySQL 或者 Oracle 的驱动包.
>
-> 如果你要添加 MySQL 数据源, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建.
-
-1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar)
-
-2. 创建一个新的 `Dockerfile`，用于添加 MySQL 驱动包:
+> 如果你要添加 MySQL 或者 Oracle, 你可以基于官方镜像 `apache/dolphinscheduler-<service>` 进行构建.
+>
+> 需要更改 dolphinscheduler-worker, dolphinscheduler-api 两个服务的镜像.
-```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
-COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
-```
+1. 下载 MySQL 驱动包 [mysql-connector-java-8.0.16.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.16/mysql-connector-java-8.0.16.jar)
+或者 Oracle 驱动包 [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (例如 `ojdbc8-19.9.0.0.jar`)
-3. 构建一个包含 MySQL 驱动包的新镜像:
+3. 创建一个新的 `Dockerfile`，用于添加 MySQL 或者 Oracle 驱动包:
```
-docker build -t apache/dolphinscheduler:mysql-driver .
-```
-
-4. 推送 docker 镜像 `apache/dolphinscheduler:mysql-driver` 到一个 docker registry 中
-
-5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `mysql-driver`
-
-6. 部署 dolphinscheduler (详见**安装 dolphinscheduler**)
-
-7. 在数据源中心添加一个 MySQL 数据源
-
-### 如何在数据源中心支持 Oracle 数据源？
-
-> 由于商业许可证的原因，我们不能直接使用 Oracle 的驱动包.
->
-> 如果你要添加 Oracle 数据源, 你可以基于官方镜像 `apache/dolphinscheduler` 进行构建.
-
-1. 下载 Oracle 驱动包 [ojdbc8.jar](https://repo1.maven.org/maven2/com/oracle/database/jdbc/ojdbc8/) (例如 `ojdbc8-19.9.0.0.jar`)
+FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-<service>:3.0.0-beta-1
+# For example
+# FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.0.0-beta-1
-2. 创建一个新的 `Dockerfile`，用于添加 Oracle 驱动包:
+# If you want to support MySQL Datasource
+COPY mysql-connector-java-8.0.16.jar /opt/dolphinscheduler/lib
-```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
+# If you want to support Oracle Datasource
COPY ojdbc8-19.9.0.0.jar /opt/dolphinscheduler/lib
```
-3. 构建一个包含 Oracle 驱动包的新镜像:
+3. 构建一个包含 MySQL 或者 Oracle 驱动包的新镜像:
```
-docker build -t apache/dolphinscheduler:oracle-driver .
+docker build -t apache/dolphinscheduler-<service>:new-driver .
```
-4. 推送 docker 镜像 `apache/dolphinscheduler:oracle-driver` 到一个 docker registry 中
+4. 推送 docker 镜像 `apache/dolphinscheduler-<service>:new-driver` 到一个 docker registry 中
-5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `oracle-driver`
+5. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `new-driver`
6. 部署 dolphinscheduler (详见**安装 dolphinscheduler**)
-7. 在数据源中心添加一个 Oracle 数据源
+7. 在数据源中心添加一个 MySQL 或者 Oracle 数据源
### 如何支持 Python 2 pip 以及自定义 requirements.txt？
+> 只需要更改 dolphinscheduler-worker 服务的镜像.
+
1. 创建一个新的 `Dockerfile`，用于安装 pip:
```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
+FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.0.0-beta-1
COPY requirements.txt /tmp
RUN apt-get update && \
apt-get install -y --no-install-recommends python-pip && \
@@ -300,16 +287,16 @@ RUN apt-get update && \
这个命令会安装默认的 **pip 18.1**. 如果你想升级 pip, 只需添加一行
```
- pip install --no-cache-dir -U pip && \
+pip install --no-cache-dir -U pip && \
```
2. 构建一个包含 pip 的新镜像:
```
-docker build -t apache/dolphinscheduler:pip .
+docker build -t apache/dolphinscheduler-worker:pip .
```
-3. 推送 docker 镜像 `apache/dolphinscheduler:pip` 到一个 docker registry 中
+3. 推送 docker 镜像 `apache/dolphinscheduler-worker:pip` 到一个 docker registry 中
4. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `pip`
@@ -319,10 +306,12 @@ docker build -t apache/dolphinscheduler:pip .
### 如何支持 Python 3？
+> 只需要更改 dolphinscheduler-worker 服务的镜像.
+
1. 创建一个新的 `Dockerfile`，用于安装 Python 3:
```
-FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler:3.0.0-beta-1
+FROM dolphinscheduler.docker.scarf.sh/apache/dolphinscheduler-worker:3.0.0-beta-1
RUN apt-get update && \
apt-get install -y --no-install-recommends python3 && \
rm -rf /var/lib/apt/lists/*
@@ -331,16 +320,16 @@ RUN apt-get update && \
这个命令会安装默认的 **Python 3.7.3**. 如果你也想安装 **pip3**, 将 `python3` 替换为 `python3-pip` 即可
```
- apt-get install -y --no-install-recommends python3-pip && \
+apt-get install -y --no-install-recommends python3-pip && \
```
2. 构建一个包含 Python 3 的新镜像:
```
-docker build -t apache/dolphinscheduler:python3 .
+docker build -t apache/dolphinscheduler-worker:python3 .
```
-3. 推送 docker 镜像 `apache/dolphinscheduler:python3` 到一个 docker registry 中
+3. 推送 docker 镜像 `apache/dolphinscheduler-worker:python3` 到一个 docker registry 中
4. 修改 `values.yaml` 文件中 image 的 `repository` 字段，并更新 `tag` 为 `python3`