You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@streampark.apache.org by be...@apache.org on 2022/12/11 08:31:09 UTC
[incubator-streampark-website] branch dev updated: Add variable management documentation (#163)
This is an automated email from the ASF dual-hosted git repository.
benjobs pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/incubator-streampark-website.git
The following commit(s) were added to refs/heads/dev by this push:
new 8dda56f0 Add variable management documentation (#163)
8dda56f0 is described below
commit 8dda56f03f29933af064ea8225a525deed54de44
Author: macksonmu <30...@qq.com>
AuthorDate: Sun Dec 11 16:31:05 2022 +0800
Add variable management documentation (#163)
* Add variable management documentation
* Add Chinese documentation for variable management
Co-authored-by: mucj7 <mu...@chinaunicom.cn>
---
docs/user-guide/7-Variable.md | 53 +++++++++++++++++++++
.../current/user-guide/7-Variable.md | 45 +++++++++++++++++
static/doc/image/variable/create_variable.png | Bin 0 -> 68605 bytes
.../variable/create_variable_desensitization.png | Bin 0 -> 70095 bytes
.../image/variable/variable_flinkjar_preview.png | Bin 0 -> 20721 bytes
.../doc/image/variable/variable_flinkjar_queto.png | Bin 0 -> 11377 bytes
.../image/variable/variable_flinksql_preview.png | Bin 0 -> 45703 bytes
.../doc/image/variable/variable_flinksql_quote.png | Bin 0 -> 32932 bytes
.../image/variable/variable_flinksql_search.png | Bin 0 -> 38860 bytes
static/doc/image/variable/variable_list.png | Bin 0 -> 74370 bytes
10 files changed, 98 insertions(+)
diff --git a/docs/user-guide/7-Variable.md b/docs/user-guide/7-Variable.md
new file mode 100644
index 00000000..d9d8cae3
--- /dev/null
+++ b/docs/user-guide/7-Variable.md
@@ -0,0 +1,53 @@
+---
+id: 'Variable'
+title: 'Variable management'
+sidebar_position: 7
+---
+
+## Background Introduction
+
+In the actual production environment, Flink jobs are generally complex, and usually require multiple external components. For example, Flink jobs consume data from Kafka, then connect external components such as HBase or Redis to obtain additional business information, and then write it to the downstream external components. There are the following problems.
+
+- The connection information of external components, such as IP, port and user password, needs to be configured in the application args and transferred to the Flink job,
+ so that the connection information of external components is distributed in multiple applications. Once the connection information of external components changes,
+ many application args parameters need to be modified, which will lead to high operation and maintenance costs.
+- There are many people in the team developing real-time computing jobs. There is no uniform specification for the connection information of the external components passed to the job,
+ resulting in different parameter names of the same component. This is difficult to count which external components are dependent.
+- In production practice, there are usually multiple sets of environments, such as test environment and
+ production environment. It is not intuitive to judge whether a component belongs to a test environment or
+ a production environment simply through IP and ports. Even if it can be judged, there are some omissions.
+ In this way, the connection information online to the production environment may be external components of the test environment,
+ or components of the production environment are inadvertently configured during testing, leading to production failures.
+
+## Create Variable
+
+Variables are isolated between teams, there are independent variables under different teams, so you need to choose which Team to create variables before creating,
+Here we choose the default team, and then click the `Add New` button to start creating variables.
+
+<img src="/doc/image/variable/create_variable.png"/><br></br>
+
+If the value of the variable is sensitive, such as a database password, you can turn on the `Desensitization` option when creating the variable, then the value of the variable will be displayed with `********` instead.
+
+<img src="/doc/image/variable/create_variable_desensitization.png"/><br></br><br></br>
+
+<img src="/doc/image/variable/variable_list.png"/><br></br>
+
+## Reference variables in Flink SQL
+
+The variables that have been created can be referenced in Flink SQL as `${kafka.cluster}`, and support follow-up searches.
+
+<img src="/doc/image/variable/variable_flinksql_search.png"/><br></br><br></br>
+
+<img src="/doc/image/variable/variable_flinksql_quote.png"/><br></br><br></br>
+
+You can click the `Preview` button to view the value of the real variable. If this variable is set as desensitized, the value of the variable will be displayed as `********`.
+
+<img src="/doc/image/variable/variable_flinksql_preview.png"/><br></br>
+
+## Reference variables in args of Flink JAR jobs
+
+Variables can also be referenced in the args of a Flink JAR job and support follow-up searches.
+
+<img src="/doc/image/variable/variable_flinkjar_queto.png"/><br></br><br></br>
+
+<img src="/doc/image/variable/variable_flinkjar_preview.png"/><br></br>
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/user-guide/7-Variable.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/user-guide/7-Variable.md
new file mode 100644
index 00000000..fa2e1fa7
--- /dev/null
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/user-guide/7-Variable.md
@@ -0,0 +1,45 @@
+---
+id: 'Variable'
+title: '变量管理'
+sidebar_position: 7
+---
+
+## 背景介绍
+
+在实际生产环境中，Flink作业一般很复杂，会依赖多个外部组件，例如，从Kafka中消费数据时要从HBase或Redis中去获取相关数据，然后将关联好的数据写入到外部组件，这样的情况下会导致如下问题：
+
+- Flink作业想要关联这些组件，需要将外部组件的连接信息传递给Flink作业，这些连接信息是跟随StreamPark的Application一起配置的，一旦一些组件的连接信息有变化，依赖这些组件的Application都要修改，这会导致大量的操作且成本很高。
+- 团队中一般有很多开发人员，如果对组件连接信息没有统一的传递规范，会导致相同的组件有不同的参数名，这样难以统计外部组件到底被哪些作业依赖。
+- 在企业生产中，通常有多个环境，比如有测试环境、生产环境等，很多时候无法通过IP和端口来判断属于哪个环境，这样的话本来属于生产环境的IP和端口可能配置到了测试环境，导致生产故障。
+
+## 创建变量
+
+变量在Team之间是隔离的，不同的Team有独立的变量管理，所以在创建变量之前首先要切换到正确的Team，这里我们选择的是default team, 通过点击`Add New`按钮创建变量。
+
+<img src="/doc/image/variable/create_variable.png"/><br></br><br></br>
+
+有些变量是敏感的，比如数据库的密码，在创建的时候可以打开`Desensitization`选项，这样变量的值将显示为`********`。
+
+<img src="/doc/image/variable/create_variable_desensitization.png"/><br></br><br></br>
+
+<img src="/doc/image/variable/variable_list.png"/><br></br>
+
+## 在Flink SQL中引用变量
+
+在Flink SQL中以`${kafka.cluster}`的方式引用变量，当输入`${`时会提示跟随搜索。
+
+<img src="/doc/image/variable/variable_flinksql_search.png"/><br></br><br></br>
+
+<img src="/doc/image/variable/variable_flinksql_quote.png"/><br></br><br></br>
+
+点击`Preview`按钮可以预览变量的值，如果变量被设置为敏感的，变量的值依然会显示为`********`。
+
+<img src="/doc/image/variable/variable_flinksql_preview.png"/><br></br>
+
+## 在Flink JAR作业的args中引用变量
+
+也可以在Flink JAR作业的args中引用变量，也支持输入`${`后跟随搜索。
+
+<img src="/doc/image/variable/variable_flinkjar_queto.png"/><br></br><br></br>
+
+<img src="/doc/image/variable/variable_flinkjar_preview.png"/><br></br>
diff --git a/static/doc/image/variable/create_variable.png b/static/doc/image/variable/create_variable.png
new file mode 100644
index 00000000..e037b911
Binary files /dev/null and b/static/doc/image/variable/create_variable.png differ
diff --git a/static/doc/image/variable/create_variable_desensitization.png b/static/doc/image/variable/create_variable_desensitization.png
new file mode 100644
index 00000000..aff17e8f
Binary files /dev/null and b/static/doc/image/variable/create_variable_desensitization.png differ
diff --git a/static/doc/image/variable/variable_flinkjar_preview.png b/static/doc/image/variable/variable_flinkjar_preview.png
new file mode 100644
index 00000000..496ac352
Binary files /dev/null and b/static/doc/image/variable/variable_flinkjar_preview.png differ
diff --git a/static/doc/image/variable/variable_flinkjar_queto.png b/static/doc/image/variable/variable_flinkjar_queto.png
new file mode 100644
index 00000000..0b01f1f2
Binary files /dev/null and b/static/doc/image/variable/variable_flinkjar_queto.png differ
diff --git a/static/doc/image/variable/variable_flinksql_preview.png b/static/doc/image/variable/variable_flinksql_preview.png
new file mode 100644
index 00000000..aecc1c07
Binary files /dev/null and b/static/doc/image/variable/variable_flinksql_preview.png differ
diff --git a/static/doc/image/variable/variable_flinksql_quote.png b/static/doc/image/variable/variable_flinksql_quote.png
new file mode 100644
index 00000000..187312a9
Binary files /dev/null and b/static/doc/image/variable/variable_flinksql_quote.png differ
diff --git a/static/doc/image/variable/variable_flinksql_search.png b/static/doc/image/variable/variable_flinksql_search.png
new file mode 100644
index 00000000..645e5fd0
Binary files /dev/null and b/static/doc/image/variable/variable_flinksql_search.png differ
diff --git a/static/doc/image/variable/variable_list.png b/static/doc/image/variable/variable_list.png
new file mode 100644
index 00000000..c68f3a25
Binary files /dev/null and b/static/doc/image/variable/variable_list.png differ