You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by he...@apache.org on 2022/03/31 01:46:35 UTC
[incubator-inlong-website] branch master updated: [INLONG-3481][Website] Optimize the standalone script (#331)
This is an automated email from the ASF dual-hosted git repository.
healchow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new ef9c754 [INLONG-3481][Website] Optimize the standalone script (#331)
ef9c754 is described below
commit ef9c7541fbf61dc7c0236d9535c2a109c23287fb
Author: dockerzhang <do...@apache.org>
AuthorDate: Thu Mar 31 09:46:28 2022 +0800
[INLONG-3481][Website] Optimize the standalone script (#331)
---
docs/deployment/bare_metal.md | 1 +
docs/deployment/standalone.md | 172 +++------------------
docs/modules/agent/quick_start.md | 3 +-
docs/modules/dataproxy/quick_start.md | 35 ++---
.../current/deployment/standalone.md | 168 +++-----------------
.../current/modules/agent/quick_start.md | 3 +-
.../current/modules/dataproxy/quick_start.md | 38 ++---
7 files changed, 80 insertions(+), 340 deletions(-)
diff --git a/docs/deployment/bare_metal.md b/docs/deployment/bare_metal.md
index befb234..eacf0b0 100644
--- a/docs/deployment/bare_metal.md
+++ b/docs/deployment/bare_metal.md
@@ -17,6 +17,7 @@ You can get binary package from [Download Page](https://inlong.apache.org/downlo
## Deploy
| order | component | dependencies | deploy guide | description |
+|-----| ---- | ---- |-----------------------------------------------------------------------| ---- |
| 1 | inlong-audit | MySQL or Elasticsearch | [InLong Audit](modules/audit/quick_start.md) | |
| 2 | inlong-manager | MySQL | [InLong Manager](modules/manager/quick_start.md) | |
| 3 | inlong-dataproxy | None | [InLong DataProxy](modules/dataproxy/quick_start.md) | |
diff --git a/docs/deployment/standalone.md b/docs/deployment/standalone.md
index 2f08b80..cb16510 100644
--- a/docs/deployment/standalone.md
+++ b/docs/deployment/standalone.md
@@ -3,169 +3,41 @@ title: Standalone
sidebar_position: 1
---
## Environment Requirements
-
-- ZooKeeper 3.5+
- MySQL 5.7+
- Flink 1.13.5
-- Apache Pulsar 2.6+ (Optional)
-- Docker or Nginx
-
-## Build
-you can refer to [How to Build](quick_start/how_to_build.md).
-
-## Configure
-
-In `conf/inlong.conf`, replace the relevant global configuration parameters according to the actual situation:
+## Prepare Message Queue
+InLong Support the following Message Queue services now, you can choose one of them.
+- [InLong TubeMQ](modules/tubemq/quick_start.md)
+- [Apache Pulsar 2.8.x](https://pulsar.apache.org/docs/en/2.8.1/standalone/)
+
+## Download the Binary Package
+You can get binary package from [Download Page](https://inlong.apache.org/download/main/) ,or you can build the InLong refer to [How to Build](quick_start/how_to_build.md).。
+
+## DB Dependencies
+- If the backend database is MySQL, please download [mysql-connector-java-8.0.27.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.27/mysql-connector-java-8.0.27.jar) and put it into the following directories:
+```bash
+inlong-agent/lib/
+inlong-audit/lib/
+inlong-manager/lib/
+inlong-tubemq-manager/lib/
```
-###############################################################################
-# Selection of message queue source: tubemq or pulsar
-source_type=tubemq
-# Selection of sort sink: hive or clickhouse
-sink_type=hive
-###############################################################################
-
-# agent
-###############################################################################
-agent_local_ip=127.0.0.1
-# whether to enable prometheus
-agent_prometheus_enable=false
-# whether to enable audit
-agent_audit_enable=true
-###############################################################################
-
-# dataproxy
-###############################################################################
-dataproxy_ip=127.0.0.1
-###############################################################################
-
-# audit proxy
-###############################################################################
-audit_proxys_ip=127.0.0.1
-###############################################################################
-
-###############################################################################
-# mysql(Required) (tubemanager,apache_inlong_manager,apache_inlong_audit)
-###############################################################################
-# MySQL service, IP, port, user and password
-spring_datasource_hostname=MYSQL_HOSTNAME
-spring_datasource_port=MYSQL_PORT
-spring_datasource_username=MYSQL_USERNAME
-spring_datasource_password=MYSQL_PASSWORD
-###############################################################################
-
-# zookeeper(Required)
-###############################################################################
-# zookeeper address
-zkserver_addr=127.0.0.1:2181
-###############################################################################
-
-# tubemq (Optional)
-###############################################################################
-# master
-# host address of master, required; must be configured at network card, enabled
-tubemaster_hostname=YOU_LOCAL_IP
-# port that master listens to, optional; default is 8715
-tubemaster_port=8715
-# port that master web console listens to
-tubemaster_webport=8080
-# metaDataPath
-metadata_path=/stage/meta_data
-# This token is used for page configuration, API call, etc
-confmod_authtoken=abc
-
-# broker
-# unique id of broker server, required; when set to 0, TubeMQ will retrieve
-# IP, convert to int and set as brokerID
-broker_id=1
-# hostname of broker server, required
-tubebroker_hostname=YOU_LOCAL_IP
-# port that broker server listens to, optional; default is 8123
-tubebroker_port=8123
-# port that broker web console listens to
-tubebroker_webport=8081
-# path to message files
-primary_path=/stage/msg_data
-
-# manager(Required)
-# the parameters for init cluster
-tube_manager_ip=YOU_LOCAL_IP
-tube_manager_port=8089
-###############################################################################
-
-# pulsar (Optional)
-###############################################################################
-# Service address of pulsar (Required)
-pulsar_admin_url=http://127.0.0.1:8080
-# Pulsar broker address（Required）
-pulsar_service_url=pulsar://127.0.0.1:6650
-# Default tenant of Pulsar
-pulsar_default_tenant=public
-###############################################################################
-
-# inlong manager
-###############################################################################
-# manager-web (Required)
-manager_server_hostname=127.0.0.1
-manager_server_port=8083
-# The default configuration file is dev
-spring_profiles_active=dev
-cluster_zk_root=inlong_hive
-# app_name-InLong Sort is cluster-id
-sort_app_name=inlong_app
-###############################################################################
+- If the backend database is PostgreSQL, there's no need for additional dependencies.
-# inlong website(Required)
-###############################################################################
-# port
-inlong_web_port=80
-# docker port
-docker_inlong_web_port=80
-###############################################################################
-
-# local_ip(Required)
-###############################################################################
-local_ip=YOU_LOCAL_IP
-###############################################################################
-```
+## Configure
+In `conf/inlong.conf`, replace the parameters according to the actual situation:
## Start
-
```shell
bin/inlong-daemon start standalone
```
-## Stop
-
-```shell
-bin/inlong-daemon stop standalone
-```
+## Register Message Queue
+You can register message queue for Manger according to [Register Step](https://inlong.apache.org/docs/next/modules/manager/quick_start#register-message-queue).
## Check
-
-After the startup script is completed, `jps -l` checks whether there are the following processes:
-
-```
-2417 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
-32658 org.apache.zookeeper.server.quorum.QuorumPeerMain
-20676 org.apache.inlong.manager.web.InLongWebApplication
-14628 org.apache.inlong.tubemq.manager.TubeMQManager
-1909 org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint
-13909 org.apache.inlong.tubemq.server.tools.MasterStartup
-12071 /root/app/apache-inlong-1.1.0-incubating-SNAPSHOT/inlong-audit/lib/audit-store-1.1.0-incubating-SNAPSHOT.jar
-19384 org.apache.inlong.dataproxy.node.Application
-20168 org.apache.inlong.agent.core.AgentMain
-14058 sun.tools.jps.Jps
-13210 org.apache.inlong.audit.node.Application
-17547 org.apache.inlong.tubemq.server.tools.BrokerStartup
-```
-If a component does not start or starts abnormally (stop and then start), you can use the following command:
-```shell
-bin/inlong-daemon start|stop agent/dataproxy/tubemaster/manager...
-```
After all component run successfully, you can access `http://localhost` with default account:
-
```shell
-User: admin
-Password: inlong
+user: admin
+password: inlong
```
\ No newline at end of file
diff --git a/docs/modules/agent/quick_start.md b/docs/modules/agent/quick_start.md
index 36ed9f1..8514683 100644
--- a/docs/modules/agent/quick_start.md
+++ b/docs/modules/agent/quick_start.md
@@ -8,8 +8,9 @@ All deploying files at `inlong-agent` directory.
## Configuration
Agent needs to pull the configuration from Manager, the configuration conf/agent.properties is as follows:
```ini
-# replace by real IP
+# replace by agent IP
agent.local.ip=127.0.0.1
+agent.http.port=8008
# manager IP
agent.manager.vip.http.host=127.0.0.1
# manager port
diff --git a/docs/modules/dataproxy/quick_start.md b/docs/modules/dataproxy/quick_start.md
index 41632b9..625d1ec 100644
--- a/docs/modules/dataproxy/quick_start.md
+++ b/docs/modules/dataproxy/quick_start.md
@@ -35,23 +35,20 @@ telnet 127.0.0.1 46801
## Add DataProxy configuration to InLong-Manager
After installing the DataProxy, you need to add the IP of the DataProxy service into the InLong-Manager.
+Modify the following information and execute command:
+```bash
+curl --header "Content-Type: application/json" --request POST http://your_manager_host:8083/api/inlong/manager/openapi/cluster/save --data '
+{
+ "name": "default_dataproxy",
+ "type": "DATA_PROXY",
+ "ip": "your_dataproxy_ip",
+ "port": 46801,
+ "mqSetName": "default_set_name",
+ "inCharges": "admin",
+ "creator": "admin"
+}
+'
+```
+- Please modify the `ip` field to the real IP (or hostname) of the DataProxy, the format is `node1:port1,node2:port2`, and separate them with `,`.
-- Modify the following information and execute command:
- ```bash
- curl --header "Content-Type: application/json" --request POST http://your_manager_host:8083/api/inlong/manager/openapi/cluster/save --data '
- {
- "name": "default_dataproxy",
- "type": "DATA_PROXY",
- "ip": "your_dataproxy_ip",
- "port": 46801,
- "mqSetName": "default_set_name",
- "inCharges": "admin",
- "creator": "admin"
- }
- '
- ```
- > **[Note]**
- >
- > Please modify the `ip` field to the real IP (or hostname) of the DataProxy, the format is `node1:port1,node2:port2`, and separate them with `,`.
- >
- > If all nodes in the `ip` field have the same port, you can use `node1,node2` and modify the `port` field to the default port for those nodes.
\ No newline at end of file
+- If all nodes in the `ip` field have the same port, you can use `node1,node2` and modify the `port` field to the default port for those nodes.
\ No newline at end of file
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/standalone.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/standalone.md
index 76a1e47..aad19a9 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/standalone.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/deployment/standalone.md
@@ -2,171 +2,45 @@
title: 单机部署
sidebar_position: 1
---
-
## 环境要求
-
-- ZooKeeper 3.5+
- MySQL 5.7+
- Flink 1.13.5
-- Apache Pulsar 2.6+ (可选)
-- Docker or Nginx
-## 编译
-可以参考[如何编译](quick_start/how_to_build.md).
+## 准备消息队列
+InLong 当前支持以下消息队列，根据使用情况**选择其一**即可。
+- [InLong TubeMQ](modules/tubemq/quick_start.md)
+- [Apache Pulsar 2.8.x](https://pulsar.apache.org/docs/en/2.8.1/standalone/)
-## 配置
+## 下载安装包
+可以从 [下载页面](https://inlong.apache.org/download/main/) 获取二进制包，或参考 [如何编译](quick_start/how_to_build.md) 编译需要的版本。
-在`conf/inlong.conf`文件中根据实际情况替换相关全局配置参数：
+## DB 依赖
+- 如果后端连接 MySQL 数据库，请下载 [mysql-connector-java-8.0.27.jar](https://repo1.maven.org/maven2/mysql/mysql-connector-java/8.0.27/mysql-connector-java-8.0.27.jar), 并将其放入以下目录：
+```bash
+inlong-agent/lib/
+inlong-audit/lib/
+inlong-manager/lib/
+inlong-tubemq-manager/lib/
```
-###############################################################################
-# Selection of message queue source: tubemq or pulsar
-source_type=tubemq
-# Selection of sort sink: hive or clickhouse
-sink_type=hive
-###############################################################################
-
-# agent
-###############################################################################
-agent_local_ip=127.0.0.1
-# whether to enable prometheus
-agent_prometheus_enable=false
-# whether to enable audit
-agent_audit_enable=true
-###############################################################################
-
-# dataproxy
-###############################################################################
-dataproxy_ip=127.0.0.1
-###############################################################################
-
-# audit proxy
-###############################################################################
-audit_proxys_ip=127.0.0.1
-###############################################################################
-
-###############################################################################
-# mysql(Required) (tubemanager,apache_inlong_manager,apache_inlong_audit)
-###############################################################################
-# MySQL service, IP, port, user and password
-spring_datasource_hostname=MYSQL_HOSTNAME
-spring_datasource_port=MYSQL_PORT
-spring_datasource_username=MYSQL_USERNAME
-spring_datasource_password=MYSQL_PASSWORD
-###############################################################################
-# zookeeper(Required)
-###############################################################################
-# zookeeper address
-zkserver_addr=127.0.0.1:2181
-###############################################################################
+- 如果后端连接 PostgreSQL 数据库，不需要引入额外依赖。
-# tubemq (Optional)
-###############################################################################
-# master
-# host address of master, required; must be configured at network card, enabled
-tubemaster_hostname=YOU_LOCAL_IP
-# port that master listens to, optional; default is 8715
-tubemaster_port=8715
-# port that master web console listens to
-tubemaster_webport=8080
-# metaDataPath
-metadata_path=/stage/meta_data
-# This token is used for page configuration, API call, etc
-confmod_authtoken=abc
-
-# broker
-# unique id of broker server, required; when set to 0, TubeMQ will retrieve
-# IP, convert to int and set as brokerID
-broker_id=1
-# hostname of broker server, required
-tubebroker_hostname=YOU_LOCAL_IP
-# port that broker server listens to, optional; default is 8123
-tubebroker_port=8123
-# port that broker web console listens to
-tubebroker_webport=8081
-# path to message files
-primary_path=/stage/msg_data
-
-# manager(Required)
-# the parameters for init cluster
-tube_manager_ip=YOU_LOCAL_IP
-tube_manager_port=8089
-###############################################################################
-
-# pulsar (Optional)
-###############################################################################
-# Service address of pulsar (Required)
-pulsar_admin_url=http://127.0.0.1:8080
-# Pulsar broker address（Required）
-pulsar_service_url=pulsar://127.0.0.1:6650
-# Default tenant of Pulsar
-pulsar_default_tenant=public
-###############################################################################
-
-# inlong manager
-###############################################################################
-# manager-web (Required)
-manager_server_hostname=127.0.0.1
-manager_server_port=8083
-# The default configuration file is dev
-spring_profiles_active=dev
-cluster_zk_root=inlong_hive
-# app_name-InLong Sort is cluster-id
-sort_app_name=inlong_app
-###############################################################################
-
-# inlong website(Required)
-###############################################################################
-# port
-inlong_web_port=80
-# docker port
-docker_inlong_web_port=80
-###############################################################################
-
-# local_ip(Required)
-###############################################################################
-local_ip=YOU_LOCAL_IP
-###############################################################################
-```
+## 配置
+在`conf/inlong.conf`文件中根据实际情况替换参数：
## 启动
-
```shell
bin/inlong-daemon start standalone
```
-## 停止
-
-```shell
-bin/inlong-daemon stop standalone
-```
+## 注册消息队列
+可参考 [注册步骤](https://inlong.apache.org/docs/next/modules/manager/quick_start#register-message-queue) 向 Manger 注册消息队列服务。
## 检查
-
-启动脚本完成后，`jps -l` 查看是否都有如下进程：
-
-```
-2417 org.apache.flink.runtime.taskexecutor.TaskManagerRunner
-32658 org.apache.zookeeper.server.quorum.QuorumPeerMain
-20676 org.apache.inlong.manager.web.InLongWebApplication
-14628 org.apache.inlong.tubemq.manager.TubeMQManager
-1909 org.apache.flink.runtime.entrypoint.StandaloneSessionClusterEntrypoint
-13909 org.apache.inlong.tubemq.server.tools.MasterStartup
-12071 /root/app/apache-inlong-1.1.0-incubating-SNAPSHOT/inlong-audit/lib/audit-store-1.1.0-incubating-SNAPSHOT.jar
-19384 org.apache.inlong.dataproxy.node.Application
-20168 org.apache.inlong.agent.core.AgentMain
-14058 sun.tools.jps.Jps
-13210 org.apache.inlong.audit.node.Application
-17547 org.apache.inlong.tubemq.server.tools.BrokerStartup
-```
-如果某组件没有启动或启动异常(停止后再启动)，可以使用尝试下面的命令：
-```shell
-bin/inlong-daemon start|stop agent/dataproxy/tubemaster/manager...
-```
当所有组件都成功启动后，可以访问`http://localhost`，并使用以下默认账号登录:
-```
-User: admin
-Password: inlong
+```shell
+user: admin
+password: inlong
```
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/agent/quick_start.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/agent/quick_start.md
index a364609..e0285d2 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/agent/quick_start.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/agent/quick_start.md
@@ -7,8 +7,9 @@ sidebar_position: 2
## 配置
Agent 需要从 Manager 拉取配置，配置conf/agent.properties如下：
```ini
-# replace by real IP
+# replace by agent IP
agent.local.ip=127.0.0.1
+agent.http.port=8008
# manager IP
agent.manager.vip.http.host=127.0.0.1
# manager port
diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/dataproxy/quick_start.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/dataproxy/quick_start.md
index c3d5727..47bbb5f 100644
--- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/dataproxy/quick_start.md
+++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/modules/dataproxy/quick_start.md
@@ -35,25 +35,19 @@ telnet 127.0.0.1 46801
## 将 DataProxy 配置添加到 InLong-Manager
-安装完 DataProxy 后，需要将 DataProxy 服务的 IP 添加到 InLong-Manager 中。
-
-- 修改下面的配置信息然后执行命令：
- ```bash
- curl --header "Content-Type: application/json" --request POST http://your_manager_host:8083/api/inlong/manager/openapi/cluster/save --data '
- {
- "name": "default_dataproxy",
- "type": "DATA_PROXY",
- "ip": "your_dataproxy_ip",
- "port": 46801,
- "mqSetName": "default_set_name",
- "inCharges": "admin",
- "creator": "admin"
- }
- '
- ```
- ```
- > **【说明】**
- >
- > 请将 `ip` 字段修改为 DataProxy 服务的真实 IP（或 hostname），格式为 `node1:port1,node2:port2`，多个时用 `,` 分隔。
- >
- > 如果 `ip` 字段中所有 node 的端口号都相同，可以使用 `node1,node2`，并将 `port` 字段修改为这些 node 的默认端口号。
\ No newline at end of file
+安装完 DataProxy 后，需要将 DataProxy 服务的 IP 添加到 InLong-Manager 中，修改下面的配置信息然后执行命令：
+```bash
+curl --header "Content-Type: application/json" --request POST http://your_manager_host:8083/api/inlong/manager/openapi/cluster/save --data '
+{
+ "name": "default_dataproxy",
+ "type": "DATA_PROXY",
+ "ip": "your_dataproxy_ip",
+ "port": 46801,
+ "mqSetName": "default_set_name",
+ "inCharges": "admin",
+ "creator": "admin"
+}
+'
+```
+- 请将 `ip` 字段修改为 DataProxy 服务的真实 IP（或 hostname），格式为 `node1:port1,node2:port2`，多个时用 `,` 分隔。
+- 如果 `ip` 字段中所有 node 的端口号都相同，可以使用 `node1,node2`，并将 `port` 字段修改为这些 node 的默认端口号。
\ No newline at end of file