You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@flink.apache.org by xt...@apache.org on 2020/07/21 06:01:14 UTC
[flink] branch release-1.11 updated (a628053 -> 4aef0b4)
This is an automated email from the ASF dual-hosted git repository.
xtsong pushed a change to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git.
from a628053 [FLINK-18635][docs] Fix typo on website
new 24d7ca5 [hotfix][doc] Change TaskExecutor to TaskManager for External Resources page.
new 90556fc [hotfix][doc] Remove redundant punctuation for External Resources page.
new fa6ace3 [hotfix][doc] Enhance the description of options for External Resources page.
new 4aef0b4 [FLINK-18264][doc-zh] Translate External Resources page to Chinese
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/ops/external_resources.md | 9 +-
docs/ops/external_resources.zh.md | 259 ++++++++++++++++++--------------------
2 files changed, 125 insertions(+), 143 deletions(-)
[flink] 01/04: [hotfix][doc] Change TaskExecutor to TaskManager for
External Resources page.
Posted by xt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
xtsong pushed a commit to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 24d7ca5daa633a477461d9b63b38eafe38808fd7
Author: Yangze Guo <ka...@gmail.com>
AuthorDate: Tue Jul 7 17:26:14 2020 +0800
[hotfix][doc] Change TaskExecutor to TaskManager for External Resources page.
---
docs/ops/external_resources.md | 2 +-
docs/ops/external_resources.zh.md | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/ops/external_resources.md b/docs/ops/external_resources.md
index c4fab362..334a82a 100644
--- a/docs/ops/external_resources.md
+++ b/docs/ops/external_resources.md
@@ -82,7 +82,7 @@ corresponds to the name listed in the **external resource list**:
- **Config key in Kubernetes** (`external-resource.<resource_name>.kubernetes.config-key`): *optional*. If configured,
external resource framework will add `resources.limits.<config-key>` and `resources.requests.<config-key>` to the main
- container spec of TaskExecutor and set the value to the value of `external-resource.<resource_name>.amount`.
+ container spec of TaskManager and set the value to the value of `external-resource.<resource_name>.amount`.
- **Driver Factory** (`external-resource.<resource_name>.driver-factory.class`): *optional*. Defines the factory class
name for the external resource identified by **\<resource_name\>**. If configured, the factory will be used to instantiate
diff --git a/docs/ops/external_resources.zh.md b/docs/ops/external_resources.zh.md
index bb9ebce..681b1fe 100644
--- a/docs/ops/external_resources.zh.md
+++ b/docs/ops/external_resources.zh.md
@@ -82,7 +82,7 @@ corresponds to the name listed in the **external resource list**:
- **Config key in Kubernetes** (`external-resource.<resource_name>.kubernetes.config-key`): *optional*. If configured,
external resource framework will add `resources.limits.<config-key>` and `resources.requests.<config-key>` to the main
- container spec of TaskExecutor and set the value to the value of `external-resource.<resource_name>.amount`.
+ container spec of TaskManager and set the value to the value of `external-resource.<resource_name>.amount`.
- **Driver Factory** (`external-resource.<resource_name>.driver-factory.class`): *optional*. Defines the factory class
name for the external resource identified by **\<resource_name\>**. If configured, the factory will be used to instantiate
[flink] 02/04: [hotfix][doc] Remove redundant punctuation for
External Resources page.
Posted by xt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
xtsong pushed a commit to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 90556fcd734516c2d9f8b9dbec02922159395f64
Author: Yangze Guo <ka...@gmail.com>
AuthorDate: Tue Jul 7 17:27:13 2020 +0800
[hotfix][doc] Remove redundant punctuation for External Resources page.
---
docs/ops/external_resources.md | 2 +-
docs/ops/external_resources.zh.md | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/docs/ops/external_resources.md b/docs/ops/external_resources.md
index 334a82a..46948d3 100644
--- a/docs/ops/external_resources.md
+++ b/docs/ops/external_resources.md
@@ -158,7 +158,7 @@ Each `ExternalResourceInfo` contains one or more properties with keys representi
You could get all valid keys by `ExternalResourceInfo#getKeys`.
<div class="alert alert-info">
- <strong>Note:</strong> Currently, the information returned by `RuntimeContext#getExternalResourceInfos` is available to all the operators.
+ <strong>Note:</strong> Currently, the information returned by RuntimeContext#getExternalResourceInfos is available to all the operators.
</div>
# Implement a plugin for your custom resource type
diff --git a/docs/ops/external_resources.zh.md b/docs/ops/external_resources.zh.md
index 681b1fe..30c9691 100644
--- a/docs/ops/external_resources.zh.md
+++ b/docs/ops/external_resources.zh.md
@@ -158,7 +158,7 @@ Each `ExternalResourceInfo` contains one or more properties with keys representi
You could get all valid keys by `ExternalResourceInfo#getKeys`.
<div class="alert alert-info">
- <strong>Note:</strong> Currently, the information returned by `RuntimeContext#getExternalResourceInfos` is available to all the operators.
+ <strong>Note:</strong> Currently, the information returned by RuntimeContext#getExternalResourceInfos is available to all the operators.
</div>
# Implement a plugin for your custom resource type
[flink] 03/04: [hotfix][doc] Enhance the description of options for
External Resources page.
Posted by xt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
xtsong pushed a commit to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git
commit fa6ace3edff9949566f4e2b8206afbd75c65f024
Author: Yangze Guo <ka...@gmail.com>
AuthorDate: Mon Jul 20 18:46:35 2020 +0800
[hotfix][doc] Enhance the description of options for External Resources page.
---
docs/ops/external_resources.md | 5 +++--
docs/ops/external_resources.zh.md | 5 +++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/docs/ops/external_resources.md b/docs/ops/external_resources.md
index 46948d3..879e435 100644
--- a/docs/ops/external_resources.md
+++ b/docs/ops/external_resources.md
@@ -86,7 +86,8 @@ corresponds to the name listed in the **external resource list**:
- **Driver Factory** (`external-resource.<resource_name>.driver-factory.class`): *optional*. Defines the factory class
name for the external resource identified by **\<resource_name\>**. If configured, the factory will be used to instantiate
- *drivers* in the external resource framework.
+ *drivers* in the external resource framework. If not configured, the requested resource will still exist in the `TaskManager`
+ as long as the relevant options are configured. However, the operator will not get any information of the resource from `RuntimeContext` in that case.
- **Driver Parameters** (`external-resource.<resource_name>.param.<param>`): *optional*. The naming pattern of custom
config options for the external resource specified by **\<resource_name\>**. Only the configurations that follow this pattern
@@ -338,7 +339,7 @@ For standalone mode, multiple TaskManagers might be co-located on the same machi
the TaskManagers. The default discovery script supports a coordination mode, in which it leverages a coordination file to
synchronize the allocation state of GPU devices and ensure each GPU device can only be used by one TaskManager process. The relevant arguments are:
- - `--enable-coordination-mode`: Enable the coordination mode.
+ - `--enable-coordination-mode`: Enable the coordination mode. By default the coordination mode is disabled.
- `--coordination-file filePath`: The path of the coordination file used to synchronize the allocation state of GPU resources. The default path is `/var/tmp/flink-gpu-coordination`.
diff --git a/docs/ops/external_resources.zh.md b/docs/ops/external_resources.zh.md
index 30c9691..74b9a4c 100644
--- a/docs/ops/external_resources.zh.md
+++ b/docs/ops/external_resources.zh.md
@@ -86,7 +86,8 @@ corresponds to the name listed in the **external resource list**:
- **Driver Factory** (`external-resource.<resource_name>.driver-factory.class`): *optional*. Defines the factory class
name for the external resource identified by **\<resource_name\>**. If configured, the factory will be used to instantiate
- *drivers* in the external resource framework.
+ *drivers* in the external resource framework. If not configured, the requested resource will still exist in the `TaskManager`
+ as long as the relevant options are configured. However, the operator will not get any information of the resource from `RuntimeContext` in that case.
- **Driver Parameters** (`external-resource.<resource_name>.param.<param>`): *optional*. The naming pattern of custom
config options for the external resource specified by **\<resource_name\>**. Only the configurations that follow this pattern
@@ -338,7 +339,7 @@ For standalone mode, multiple TaskManagers might be co-located on the same machi
the TaskManagers. The default discovery script supports a coordination mode, in which it leverages a coordination file to
synchronize the allocation state of GPU devices and ensure each GPU device can only be used by one TaskManager process. The relevant arguments are:
- - `--enable-coordination-mode`: Enable the coordination mode.
+ - `--enable-coordination-mode`: Enable the coordination mode. By default the coordination mode is disabled.
- `--coordination-file filePath`: The path of the coordination file used to synchronize the allocation state of GPU resources. The default path is `/var/tmp/flink-gpu-coordination`.
[flink] 04/04: [FLINK-18264][doc-zh] Translate External Resources
page to Chinese
Posted by xt...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
xtsong pushed a commit to branch release-1.11
in repository https://gitbox.apache.org/repos/asf/flink.git
commit 4aef0b4db01eac6963aedda36fbb962b308067a0
Author: Yangze Guo <ka...@gmail.com>
AuthorDate: Wed Jun 24 16:51:30 2020 +0800
[FLINK-18264][doc-zh] Translate External Resources page to Chinese
This closes #12865.
---
docs/ops/external_resources.zh.md | 260 ++++++++++++++++++--------------------
1 file changed, 120 insertions(+), 140 deletions(-)
diff --git a/docs/ops/external_resources.zh.md b/docs/ops/external_resources.zh.md
index 74b9a4c..15a5a27 100644
--- a/docs/ops/external_resources.zh.md
+++ b/docs/ops/external_resources.zh.md
@@ -1,8 +1,8 @@
---
-title: "外部资源调度框架"
+title: "扩展资源框架"
nav-parent_id: ops
nav-pos: 10
-nav-title: External Resources
+nav-title: 扩展资源
---
<!--
Licensed to the Apache Software Foundation (ASF) under one
@@ -23,99 +23,88 @@ specific language governing permissions and limitations
under the License.
-->
-In addition to CPU and memory, many workloads also need some other resources, e.g. GPUs for deep learning. To support external
-resources, Flink provides an external resource framework. The framework supports requesting various types of resources from the
-underlying resource management systems (e.g., Kubernetes), and supplies information needed for using these resources to the operators.
-Different resource types can be supported. You can either leverage built-in plugins provided by Flink (currently only for GPU support),
-or implement your own plugins for custom resource types.
+许多计算任务需要使用除了 CPU 与内存外的资源，如用深度学习场景需要使用 GPU 来进行加速。为了支持这种扩展资源，Flink 提供了一个扩展资源框架。
+该框架支持从底层资源管理系统（如 Kubernetes）请求各种类型的资源，并向算子提供使用这些资源所需的信息。该框架以插件形式支持不同的资源类型。
+目前 Flink 仅内置了支持 GPU 资源的插件，你可以为你想使用的资源类型实现第三方插件。
* This will be replaced by the TOC
{:toc}
-# What the external resource framework does
+<a name="what-the-external-resource-framework-does"></a>
-In general, the external resource framework does two things:
+# 扩展资源框架做了什么
- - Set the corresponding fields of the resource requests (for requesting resources from the underlying system) with respect to your configuration.
+扩展资源（External Resource）框架主要做了以下两件事：
- - Provide operators with the *information* needed for using the resources.
+ - 根据你的配置，在 Flink 从底层资源管理系统中申请资源时，设置与扩展资源相关的请求字段
-When deployed on resource management systems (Kubernetes / Yarn), the external resource framework will ensure that the allocated pod/container
-will contain the desired external resources. Currently, many resource management systems support external resources. For example,
-Kubernetes supports GPU, FPGA, etc. through its [Device Plugin](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/)
-mechanism since v1.10, and Yarn supports GPU and FPGA resources since 2.10 and 3.1. External resources are not supported by Flink’s Mesos
-integration at the moment. In Standalone mode, the user has to ensure that the external resources are available.
+ - 为算子提供使用这些资源所需要的*信息*
-The external resource framework will provide the corresponding *information* to operators. The external resource information,
-which contains the basic properties needed for using the resources, is generated by the configured external resource *drivers*.
+当 Flink 部署在资源管理系统（Kubernetes、Yarn）上时，扩展资源框架将确保分配的 Pod、Container 包含所需的扩展资源。目前，许多资源管理系统都支持扩展资源。
+例如，Kubernetes 从 v1.10 开始通过 [Device Plugin](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) 机制支持 GPU、FPGA 等资源调度，Yarn 从 2.10 和 3.1 开始支持 GPU 和 FPGA 的调度。
+目前，扩展资源框架并不支持 Mesos 模式。在 Standalone 模式下，由用户负责确保扩展资源的可用性。
-# Enable the external resource framework for your workload
+扩展资源框架向算子提供扩展资源相关*信息*，这些信息由你配置的扩展资源 *Driver* 生成，包含了使用扩展资源所需要的基本属性。
-To enable an external resource with the external resource framework, you need to:
+<a name="enable-the-external-resource-framework-for-your-workload"></a>
- - Prepare the external resource *plugin*.
+# 启用扩展资源框架
- - Set configurations for the external resource.
+为了启用扩展资源框架来使用扩展资源，你需要：
- - Get the external resource *information* from `RuntimeContext` and use it in your operators.
+ - 为该扩展资源准备扩展资源框架的*插件*
-## Prepare plugins
+ - 为该扩展资源设置相关的配置
-You need to prepare the external resource plugin and put it into the `plugins/` folder of your Flink distribution, see
-[Flink Plugins]({% link ops/plugins.zh.md %}). Apache Flink provides a first-party [plugin for GPU resources](#plugin-for-gpu-resources). You can also
-[implement a plugin for your custom resource type](#implement-a-plugin-for-your-custom-resource-type).
+ - 在你的算子中，从 `RuntimeContext` 来获取扩展资源的*信息*并使用这些资源
-## Configurations
+<a name="prepare-plugins"></a>
-First, you need to add resource names for all the external resource types to the **external resource list (with the configuration key ‘external-resources’)**
-with delimiter ";", e.g. "external-resources: gpu;fpga" for two external resources "gpu" and "fpga". Only the **\<resource_name\>**
-defined here will go into effect in the external resource framework.
+## 准备插件
-For each external resource, you could configure the below options. The **\<resource_name\>** in all the below configuration options
-corresponds to the name listed in the **external resource list**:
+你需要为使用的扩展资源准备插件，并将其放入 Flink 发行版的 `plugins/` 文件夹中, 参看 [Flink Plugins]({% link ops/plugins.zh.md %})。
+Flink 提供了第一方的 [GPU 资源插件](#plugin-for-gpu-resources)。你同样可以为你所使用的扩展资源实现自定义插件[实现自定义插件](#implement-a-plugin-for-your-custom-resource-type)。
- - **Amount** (`external.<resource_name>.amount`): This is the quantity of the external resource that should be requested from the external system.
+## 配置项
- - **Config key in Yarn** (`external-resource.<resource_name>.yarn.config-key`): *optional*. If configured, the external
- resource framework will add this key to the resource profile of container requests for Yarn. The value will be set to the
- value of `external-resource.<resource_name>.amount`.
+首先，你需要使用分隔符“;”将所有使用的扩展资源类型的资源名称添加到**扩展资源列表（配置键“external-resources”）**中，例如，“external-resources: gpu;fpga”定义了两个扩展资源“gpu”和“fpga”。
+只有此处定义了扩展资源名称（**\<resource_name\>**），相应的资源才会在扩展资源框架中生效。
- - **Config key in Kubernetes** (`external-resource.<resource_name>.kubernetes.config-key`): *optional*. If configured,
- external resource framework will add `resources.limits.<config-key>` and `resources.requests.<config-key>` to the main
- container spec of TaskManager and set the value to the value of `external-resource.<resource_name>.amount`.
+对于每个扩展资源，有以下配置选项。下面的所有配置选项中的 **\<resource_name\>** 对应于**扩展资源列表**中列出的名称：
- - **Driver Factory** (`external-resource.<resource_name>.driver-factory.class`): *optional*. Defines the factory class
- name for the external resource identified by **\<resource_name\>**. If configured, the factory will be used to instantiate
- *drivers* in the external resource framework. If not configured, the requested resource will still exist in the `TaskManager`
- as long as the relevant options are configured. However, the operator will not get any information of the resource from `RuntimeContext` in that case.
+ - **数量** （`external.<resource_name>.amount`）：需要从外部系统请求的扩展资源的数量。
- - **Driver Parameters** (`external-resource.<resource_name>.param.<param>`): *optional*. The naming pattern of custom
- config options for the external resource specified by **\<resource_name\>**. Only the configurations that follow this pattern
- will be passed into the *driver* factory of that external resource.
+ - **Yarn 中的扩展资源配置键** （`external-resource.<resource_name>.yarn.config-key`）：*可选配置*。如果配置该项，扩展资源框架将把这个键添加到 Yarn 的容器请求的资源配置中，该键对应的值将被设置为`external-resource.<resource_name>.amount`。
-An example configuration that specifies two external resources:
+ - **Kubernetes 中的扩展资源配置键** （`external-resource.<resource_name>.kubernetes.config-key`）：*可选配置*。
+ 如果配置该项，扩展资源框架将添加 `resources.limits.<config-key>` 和 `resources.requests.<config-key>` 到 TaskManager 的主容器配置中，对应的值将被设置为 `external-resource.<resource_name>.amount`。
+
+ - **Driver 工厂类** （`external-resource.<resource_name>.driver-factory.class`）：*可选配置*。定义由 **\<resource_name\>** 标识的扩展资源对应的工厂类名。如果配置该项，该工厂类将被用于实例化扩展资源框架中所需要的 *drivers*。
+ 如果没有配置，扩展资源依然会在其他配置正确时在存在于 `TaskManager`，只是算子在这种情况下无法从 `RuntimeContext` 中拿到该资源的信息。
+
+ - **Driver 自定义参数** （`external-resource.<resource_name>.param.<param>`）：*可选配置*。由 **\<resource_name\>** 标识的扩展资源的自定义配置选项的命名模式。只有遵循此模式的配置才会传递到该扩展资源的工厂类。
+
+示例配置，该配置定义两个扩展资源：
{% highlight bash %}
-external-resources: gpu;fpga # Define two external resources, "gpu" and "fpga".
+external-resources: gpu;fpga # 定义两个扩展资源，“gpu”和“fpga”。
-external-resource.gpu.driver-factory.class: org.apache.flink.externalresource.gpu.GPUDriverFactory # Define the driver factory class of gpu resource.
-external-resource.gpu.amount: 2 # Define the amount of gpu resource per TaskManager.
-external-resource.gpu.param.discovery-script.args: --enable-coordination # Define the custom param discovery-script.args which will be passed into the gpu driver.
+external-resource.gpu.driver-factory.class: org.apache.flink.externalresource.gpu.GPUDriverFactory # 定义 GPU 资源对应 Driver 的工厂类。
+external-resource.gpu.amount: 2 # 定义每个 TaskManager 所需的 GPU 数量。
+external-resource.gpu.param.discovery-script.args: --enable-coordination # 自定义参数 discovery-script.args，它将被传递到 GPU 对应的 Driver 中。
-external-resource.fpga.driver-factory.class: org.apache.flink.externalresource.fpga.FPGADriverFactory # Define the driver factory class of fpga resource.
-external-resource.fpga.amount: 1 # Define the amount of fpga resource per TaskManager.
-external-resource.fpga.yarn.config-key: yarn.io/fpga # Define the corresponding config key of fpga in Yarn.
+external-resource.fpga.driver-factory.class: org.apache.flink.externalresource.fpga.FPGADriverFactory # 定义 FPGA 资源对应 Driver 的工厂类。
+external-resource.fpga.amount: 1 # 定义每个 TaskManager 所需的 FPGA 数量。
+external-resource.fpga.yarn.config-key: yarn.io/fpga # 定义 FPGA 在 Yarn 中对应的配置键。
{% endhighlight %}
-## Use the resources
+## 使用扩展资源
-To use the resources, operators need to get the `ExternalResourceInfo` set from the `RuntimeContext`. `ExternalResourceInfo`
-wraps the information needed for using the resource, which can be retrieved with `getProperty`. What properties are available
-and how to access the resource with the properties depends on the specific plugin.
+为了使用扩展资源，算子需要从 `RuntimeContext` 获取 `ExternalResourceInfo` 集合。 `ExternalResourceInfo` 包含了使用扩展资源所需的信息，可以使用 `getProperty` 检索这些信息。
+其中具体包含哪些属性以及如何使用这些属性取决于特定的扩展资源插件。
-Operators can get the `ExternalResourceInfo` set of a specific external resource from `RuntimeContext` or `FunctionContext` by
-`getExternalResourceInfos(String resourceName)`. The `resourceName` here should have the same value as the name configured in the
-external resource list. It can be used as follows:
+算子可以通过 `getExternalResourceInfos(String resourceName)` 从 `RuntimeContext` 或 `FunctionContext` 中获取特定扩展资源的 `ExternalResourceInfo`。
+此处的 `resourceName` 应与在扩展资源列表中定义的名称相同。具体用法如下：
<div class="codetabs" markdown="1">
<div data-lang="java" markdown="1">
@@ -155,25 +144,25 @@ class ExternalResourceMapFunction extends RichMapFunction[(String, String)] {
</div>
</div>
-Each `ExternalResourceInfo` contains one or more properties with keys representing the different dimensions of the resource.
-You could get all valid keys by `ExternalResourceInfo#getKeys`.
+`ExternalResourceInfo` 中包含一个或多个键-值对，其键值表示资源的不同维度。你可以通过 `ExternalResourceInfo#getKeys` 获取所有的键。
<div class="alert alert-info">
- <strong>Note:</strong> Currently, the information returned by RuntimeContext#getExternalResourceInfos is available to all the operators.
+ <strong>提示：</strong>目前，RuntimeContext#getExternalResourceInfos 返回的信息对所有算子都是可用的。
</div>
-# Implement a plugin for your custom resource type
+<a name="implement-a-plugin-for-your-custom-resource-type"></a>
+
+# 为你所使用的扩展资源实现自定义插件
-To implement a plugin for your custom resource type, you need to:
+要为你所使用的扩展资源实现自定义插件，你需要：
- - Add your own external resource driver by implementing the `org.apache.flink.api.common.externalresource.ExternalResourceDriver` interface.
+ - 添加你自定义的扩展资源 Driver ，该 Driver 需要实现 `org.apache.flink.api.common.externalresource.ExternalResourceDriver` 接口。
- - Add a driver factory, which instantiates the *driver*, by implementing the `org.apache.flink.api.common.externalresource.ExternalResourceDriverFactory`.
+ - 添加用来实例化 *Driver* 的工厂类，该工厂类需要实现 `org.apache.flink.api.common.externalresource.ExternalResourceDriverFactory` 接口。
- - Add a service entry. Create a file `META-INF/services/org.apache.flink.api.common.externalresource.ExternalResourceDriverFactory`
- which contains the class name of your *driver* factory class (see the [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html) docs for more details).
+ - 添加服务入口。创建 `META-INF/services/org.apache.flink.api.common.externalresource.ExternalResourceDriverFactory` 文件，其中包含了 *Driver* 对应工厂类的类名（更多细节请参看 [Java Service Loader](https://docs.oracle.com/javase/8/docs/api/java/util/ServiceLoader.html)）。
-For example, to implement a plugin for external resource named "FPGA", you need to implement `FPGADriver` and `FPGADriverFactory` first:
+例如，要为名为“FPGA”的扩展资源实现插件，你首先需要实现 `FPGADriver` 和 `FPGADriverFactory`：
<div class="codetabs" markdown="1">
<div data-lang="java" markdown="1">
@@ -235,127 +224,118 @@ class FPGAInfo extends ExternalResourceInfo {
</div>
</div>
-Create a file with name `org.apache.flink.api.common.externalresource.ExternalResourceDriverFactory` in `META-INF/services/`
-and write the factory class name (e.g. `your.domain.FPGADriverFactory`) to it.
+在 `META-INF/services/` 中创建名为 `org.apache.flink.api.common.externalresource.ExternalResourceDriverFactory` 的文件，向其中写入工厂类名，如 `your.domain.FPGADriverFactory`。
-Then, create a jar which includes `FPGADriver`, `FPGADriverFactory`, `META-INF/services/` and all the external dependencies.
-Make a directory in `plugins/` of your Flink distribution with an arbitrary name, e.g. "fpga", and put the jar into this directory.
-See [Flink Plugin]({% link ops/plugins.zh.md %}) for more details.
+之后，将 `FPGADriver`，`FPGADriverFactory`，`META-INF/services/` 和所有外部依赖打入 jar 包。在你的 Flink 发行版的 `plugins/` 文件夹中创建一个名为“fpga”的文件夹，将打好的 jar 包放入其中。
+更多细节请查看 [Flink Plugin]({% link ops/plugins.zh.md %})。
<div class="alert alert-info">
- <strong>Note:</strong> External resources are shared by all operators running on the same machine. The community might add external resource isolation in a future release.
+ <strong>提示：</strong> 扩展资源由运行在同一台机器上的所有算子共享。社区可能会在未来的版本中支持外部资源隔离。
</div>
-# Existing supported external resource plugins
+# 已支持的扩展资源插件
-Currently, Flink supports GPUs as external resources.
+目前，Flink提供 GPU 资源插件。
-## Plugin for GPU resources
+<a name="plugin-for-gpu-resources"></a>
-We provide a first-party plugin for GPU resources. The plugin leverages a discovery script to discover indexes of GPU devices, which can
-be accessed from the resource *information* via the property "index". We provide a default discovery script that can be used to discover
-NVIDIA GPUs. You can also provide your custom script.
+## GPU 插件
-We provide [an example](https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/gpu/MatrixVectorMul.java)
-which shows how to use the GPUs to do matrix-vector multiplication in Flink.
+我们为 GPU 提供了第一方插件。该插件利用一个脚本来发现 GPU 设备的索引，该索引可通过“index”从 `ExternalResourceInfo` 中获取。我们提供了一个默认脚本，可以用来发现 NVIDIA GPU。您还可以提供自定义脚本。
+
+我们提供了[一个示例程序](https://github.com/apache/flink/blob/master/flink-examples/flink-examples-streaming/src/main/java/org/apache/flink/streaming/examples/gpu/MatrixVectorMul.java)，展示了如何在 Flink 中使用 GPU 资源来做矩阵-向量乘法。
<div class="alert alert-info">
- <strong>Note:</strong> Currently, for all the operators, RuntimeContext#getExternalResourceInfos returns the same set of resource information. That means, the same set of GPU devices are always accessible to all the operators running in the same TaskManager. There is no operator level isolation at the moment.
+ <strong>提示：</strong>目前，对于所有算子，RuntimeContext#getExternalResourceInfos 会返回同样的资源信息。也即，在同一个 TaskManager 中运行的所有算子都可以访问同一组 GPU 设备。扩展资源目前没有算子级别的隔离。
</div>
-### Pre-requisites
+### 前置准备
-To make GPU resources accessible, certain prerequisites are needed depending on your environment:
+要使 GPU 资源可访问，根据您的环境，需要满足以下先决条件：
- - For standalone mode, administrators should ensure the NVIDIA driver is installed and GPU resources are accessible on all the nodes in the cluster.
+ - 对于 Standalone 模式，集群管理员应确保已安装 NVIDIA 驱动程序，并且集群中所有节点上的 GPU 资源都是可访问的。
- - For Yarn deployment, administrators should configure the Yarn cluster to enable [GPU scheduling](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/UsingGpus.html).
- Notice the required Hadoop version is 2.10+ or 3.1+.
+ - 对于 Yarn 上部署，管理员需要配置 Yarn 集群使其[支持 GPU 调度](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/UsingGpus.html)。
+ 请注意，所需的 Hadoop 版本是 2.10+ 和 3.1+。
- - For Kubernetes deployment, administrators should make sure the NVIDIA GPU [device plugin](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/)
- is installed. Notice the required version is 1.10+. At the moment, Kubernetes only supports NVIDIA GPU and AMD GPU. Flink only provides discovery script for NVIDIA GPUs,
- but you can provide a custom discovery script for AMD GPUs yourself, see [Discovery script](#discovery-script).
+ - 对于 Kubernetes 上部署，管理员需要保证 NVIDIA GPU 的 [Device Plugin](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) 已在集群上安装。
+ 请注意，所需的 Kubernetes 版本是 1.10+。目前，Kubernetes只支持 NVIDIA GPU 和 AMD GPU。Flink 只提供了 NVIDIA GPU 的脚本，但你可以提供支持 AMD GPU 的自定义脚本，参看 [发现脚本](#discovery-script)。
-### Enable GPU resources for your workload
+### 在计算任务中使用 GPU 资源
-As mentioned in [Enable external resources for your workload](#enable-the-external-resource-framework-for-your-workload),
-you also need to do two things to enable GPU resources:
+如[启用扩展资源框架](#enable-the-external-resource-framework-for-your-workload)中所述，要使用 GPU 资源，还需要执行两项操作：
- - Configure the GPU resource.
+ - 为 GPU 资源进行相关配置。
- - Get the *information* of GPU resources, which contains the GPU index as property with key "index", in operators.
+ - 在算子中获取 GPU 资源的*信息*，其中包含键为“index”的 GPU 索引。
-### Configurations
+### 配置项
-For the GPU plugin, you need to specify the common external resource configurations:
+对于 GPU 插件，你需要指定的扩展资源框架配置：
- - `external-resources`: You need to append your resource name (e.g. gpu) for GPU resources to it.
+ - `external-resources`：你需要将 GPU 的扩展资源名称（例如“gpu”）加到该列表中。
- - `external-resource.<resource_name>.amount`: The amount of GPU devices per TaskManager.
+ - `external-resource.<resource_name>.amount`：每个 TaskManager 中的 GPU 数量。
- - `external-resource.<resource_name>.yarn.config-key`: For Yarn, the config key of GPU is `yarn.io/gpu`. Notice that
- Yarn only supports NVIDIA GPU at the moment.
+ - `external-resource.<resource_name>.yarn.config-key`：对于 Yarn，GPU 的配置键是 `yarn.io/gpu`。请注意，Yarn 目前只支持 NVIDIA GPU。
- - `external-resource.<resource_name>.kubernetes.config-key`: For Kubernetes, the config key of GPU is `<vendor>.com/gpu`.
- Currently, "nvidia" and "amd" are the two supported vendors. Notice that if you use AMD GPUs, you need to provide a discovery
- script yourself, see [Discovery script](#discovery-script).
+ - `external-resource.<resource_name>.kubernetes.config-key`：对于 Kubernetes，GPU 的配置键是 `<vendor>.com/gpu`。
+ 目前，“nvidia”和“amd”是两个支持的 GPU 品牌。请注意，如果你使用 AMD GPU，你需要提供一个自定义的[发现脚本](#discovery-script)。
- - external-resource.<resource_name>.driver-factory.class: Should be set to org.apache.flink.externalresource.gpu.GPUDriverFactory.
+ - `external-resource.<resource_name>.driver-factory.class`：需要设置为 org.apache.flink.externalresource.gpu.GPUDriverFactory。
-In addition, there are some specific configurations for the GPU plugin:
+此外，GPU 插件还有一些专有配置：
- - `external-resource.<resource_name>.param.discovery-script.path`: The path of the [discovery script](#discovery-script). It
- can either be an absolute path, or a relative path to `FLINK_HOME` when defined or current directory otherwise. If not
- explicitly configured, the default script will be used.
+ - `external-resource.<resource_name>.param.discovery-script.path`：[发现脚本](#discovery-script)的文件路径。
+ 它既可以是绝对路径，也可以是相对路径，如果定义了“FLINK_HOME”，该路径将相对于“FLINK_HOME”，否则相对于当前目录。如果没有显式配置该项，GPU 插件将使用默认脚本。
- - `external-resource.<resource_name>.param.discovery-script.args`: The arguments passed to the discovery script. For the default
- discovery script, see [Default Script](#default-script) for the available parameters.
+ - `external-resource.<resource_name>.param.discovery-script.args`：传递给发现脚本的参数。对于默认的发现脚本，请参见[默认脚本](#default-script)以获取可用参数。
-An example configuration for GPU resource:
+GPU 插件示例配置：
{% highlight bash %}
external-resources: gpu
-external-resource.gpu.driver-factory.class: org.apache.flink.externalresource.gpu.GPUDriverFactory # Define the driver factory class of gpu resource.
-external-resource.gpu.amount: 2 # Define the amount of gpu resource per TaskManager.
+external-resource.gpu.driver-factory.class: org.apache.flink.externalresource.gpu.GPUDriverFactory # 定义 GPU 资源的工厂类。
+external-resource.gpu.amount: 2 # 定义每个 TaskManager 的 GPU 数量。
external-resource.gpu.param.discovery-script.path: plugins/external-resource-gpu/nvidia-gpu-discovery.sh
-external-resource.gpu.param.discovery-script.args: --enable-coordination # Define the custom param "discovery-script.args" which will be passed into the gpu driver.
+external-resource.gpu.param.discovery-script.args: --enable-coordination # 自定义参数，将被传递到 GPU 的 Driver 中。
external-resource.gpu.yarn.config-key: yarn.io/gpu # for Yarn
external-resource.gpu.kubernetes.config-key: nvidia.com/gpu # for Kubernetes
{% endhighlight %}
-### Discovery script
+<a name="discovery-script"></a>
+
+### 发现脚本
+
+`GPUDriver` 利用发现脚本来发现 GPU 资源并生成 GPU 资源信息。
-The `GPUDriver` leverages a discovery script to discover GPU resources and generate the GPU resource information.
+<a name="default-script"></a>
-#### Default Script
+#### 默认脚本
-We provide a default discovery script for NVIDIA GPU, located at `plugins/external-resource-gpu/nvidia-gpu-discovery.sh` of your
-Flink distribution. The script gets the indexes of visible GPU resources through the `nvidia-smi` command. It tries to return
-the required amount (specified by `external-resource.<resource_name>.amount`) of GPU indexes in a list, and exit with non-zero if the amount cannot be satisfied.
+我们为 NVIDIA GPU 提供了一个默认脚本，位于 Flink 发行版的 `plugins/external-resource-gpu/nvidia-gpu-discovery.sh`。
+该脚本通过 `nvidia-smi` 工具获取当前可见 GPU 的索引。它尝试返回一个 GPU 索引列表，其大小由 `external-resource.<resource_name>.amount` 指定，如果 GPU 数量不足，则以非零退出。
-For standalone mode, multiple TaskManagers might be co-located on the same machine, and each GPU device is visible to all
-the TaskManagers. The default discovery script supports a coordination mode, in which it leverages a coordination file to
-synchronize the allocation state of GPU devices and ensure each GPU device can only be used by one TaskManager process. The relevant arguments are:
+在 Standalone 模式中，多个 TaskManager 可能位于同一台机器上，并且每个 GPU 设备对所有 TaskManager 都是可见的。
+默认脚本提供 GPU 协调模式，在这种模式下，脚本利用文件来同步 GPU 的分配情况，并确保每个GPU设备只能由一个TaskManager进程使用。相关参数为：
- - `--enable-coordination-mode`: Enable the coordination mode. By default the coordination mode is disabled.
+ - `--enable-coordination-mode`：启用 GPU 协调模式。默认情况下不启用。
- - `--coordination-file filePath`: The path of the coordination file used to synchronize the allocation state of GPU resources. The default path is `/var/tmp/flink-gpu-coordination`.
+ - `--coordination-file filePath`：用于同步 GPU 资源分配状态的文件路径。默认路径为 `/var/tmp/flink-gpu-coordination`。
<div class="alert alert-info">
- <strong>Note:</strong> The coordination mode only ensures that a GPU device is not shared by multiple TaskManagers of the same Flink cluster. Please be aware that another Flink cluster (with a different coordination file) or a non-Flink application can still use the same GPU devices.
+ <strong>提示：</strong>协调模式只确保一个 GPU 设备不会被同一个 Flink 集群的多个 TaskManager 共享。不同 Flink 集群间（具有不同的协调文件）或非 Flink 应用程序仍然可以使用相同的 GPU 设备。
</div>
-#### Custom Script
+#### 自定义脚本
-You can also provide a discovery script to address your custom requirements, e.g. discovering AMD GPU. Please make sure
-the path of your custom script is accessible to Flink and configured (`external-resource.<resource_name>.param.discovery-script.path`) correctly.
-The contract of the discovery script:
+你可以提供一个自定义的发现脚本来满足你的特殊需求，例如使用 AMD GPU。请确保自定义脚本的的路径正确配置（`external-resource.<resource_name>.param.discovery-script.path`）并且 Flink 可以访问。自定义的发现脚本需要：
- - `GPUDriver` passes the amount (specified by `external-resource.<resource_name>.amount`) as the first argument into the script.
- The user-defined arguments in `external-resource.<resource_name>.param.discovery-script.args` would be appended after it.
+ - `GPUDriver` 将 GPU 数量（由 `external-resource.<resource_name>.amount` 定义）作为第一个参数传递到脚本中。
+ `external-resource.<resource_name>.param.discovery-script.args` 中自定义的参数会被附加在后面。
- - The script should return a list of the available GPU indexes, split by a comma. Whitespace only indexes will be ignored.
+ - 脚本需返回可用 GPU 索引的列表，用逗号分隔。空白的索引将被忽略。
- - The script can also suggest that the discovery is not properly performed, by exiting with non-zero. In that case, no gpu information will be provided to operators.
+ - 脚本可以通过以非零退出来表示其未正确执行。在这种情况下，算子将不会得到 GPU 资源相关信息。