You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by go...@apache.org on 2021/07/07 04:52:07 UTC
[incubator-inlong-website] branch master updated: [INLONG-665]
Adjust TubeMQ's quick_start.md content (#79)
This is an automated email from the ASF dual-hosted git repository.
gosonzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 41f2275 [INLONG-665] Adjust TubeMQ's quick_start.md content (#79)
41f2275 is described below
commit 41f2275a631173fad89ae40e317d4e2e8e95f918
Author: gosonzhang <go...@apache.org>
AuthorDate: Wed Jul 7 12:52:00 2021 +0800
[INLONG-665] Adjust TubeMQ's quick_start.md content (#79)
---
docs/en-us/modules/tubemq/quick_start.md | 84 +++++++++++--------------
docs/en-us/quick_start.md | 4 +-
docs/zh-cn/modules/tubemq/quick_start.md | 101 ++++++++++++++-----------------
docs/zh-cn/quick_start.md | 4 +-
site_config/docs.js | 20 ++++--
5 files changed, 101 insertions(+), 112 deletions(-)
diff --git a/docs/en-us/modules/tubemq/quick_start.md b/docs/en-us/modules/tubemq/quick_start.md
index 6f3a1fc..315ad69 100644
--- a/docs/en-us/modules/tubemq/quick_start.md
+++ b/docs/en-us/modules/tubemq/quick_start.md
@@ -3,8 +3,9 @@ title: Quick Start - Apache InLong's TubeMQ module
---
## Build TubeMQ
+
### Prerequisites
-- Java JDK 1.7 or 1.8
+- Java JDK 1.8
- Maven 3.3+
### Build Distribution Tarball
@@ -12,35 +13,31 @@ title: Quick Start - Apache InLong's TubeMQ module
```bash
mvn clean package -DskipTests
```
-- (Optional) Build Using Docker：
-```bash
-docker run -v REPLACE_WITH_SOURCE_PATH:/tubemq apachetubemq/tubemq-build clean package -DskipTests
-```
+
- Run Unit Tests：
```bash
mvn test
```
+
- Build Individual Module：
```bash
mvn clean install
cd module-name (e.g. tubemq-client)
mvn test
```
+
After the build, please go to `tubemq-server/target`. You can find the
**apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin.tar.gz** file. It is the TubeMQ deployment package, which includes
scripts, configuration files, dependency jars and web GUI code.
### Setting Up Your IDE
If you want to build and debug source code in IDE, go to the project root, and run
-
```bash
mvn compile
```
-
This command will generate the Java source files from the `protoc` files, the generated files located in `target/generated-sources`.
(Optional) If you want to use local `protoc` executable, you can change the configuration of `protobuf-maven-plugin` in `tubemq-core/pom.xml` as below
-
```xml
<configuration>
<outputDirectory>${project.build.directory}/generated-sources/java</outputDirectory>
@@ -49,17 +46,18 @@ This command will generate the Java source files from the `protoc` files, the ge
```
## Deploy and Start
-### Deploy TubeMQ Standalone
-Standalone mode starts zookeeper/master/broker services in one docker container：
-```
-docker run -p 8080:8080 -p 8000:8000 -p 8123:8123 --name tubemq -d apachetubemq/tubemq-all:latest
-```
-Afater container is running, you can access ` http://127.0.0.1:8080`, and reference to next `Quick Start` chapter for experience.
-**Tips**：Standalone Mode is only available for development and experience, it's not designed for production environment.
+### Configuration Example
+There're two components in the cluster: **Master** and **Broker**. Master and Broker
+can be deployed on the same server or different servers. In this example, we setup our cluster
+like this, and all services run on the same node. Zookeeper should be setup in your environment also.
+| Role | TCP Port | TLS Port | Web Port | Comment |
+| ---- | -------- | -------- | -------- | ------- |
+| Master | 8099 | 8199 | 8080 | Meta data is stored at /stage/meta_data |
+| Broker | 8123 | 8124 | 8081 | Message is stored at /stage/msg_data |
+| Zookeeper | 2181 | | | Offset is stored at /tubemq |
-### Deploy TubeMQ Cluster
-#### Prerequisites
+### Prerequisites
- ZooKeeper Cluster
- [apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin.tar.gz](download/download.md) package file
@@ -72,18 +70,8 @@ After you extract the package file, here's the folder structure.
├── logs
└── resources
```
-#### Configuration Example
-There're two components in the cluster: **Master** and **Broker**. Master and Broker
-can be deployed on the same server or different servers. In this example, we setup our cluster
-like this, and all services run on the same node. Zookeeper should be setup in your environment also.
-
-| Role | TCP Port | TLS Port | Web Port | Comment |
-| ---- | -------- | -------- | -------- | ------- |
-| Master | 8099 | 8199 | 8080 | Meta data is stored at /stage/meta_data |
-| Broker | 8123 | 8124 | 8081 | Message is stored at /stage/msg_data |
-| Zookeeper | 2181 | | | Offset is stored at /tubemq |
-#### Configure Master
+### Configure Master
You can change configurations in `conf/master.ini` according to cluster information.
- Master IP and Port
```ini
@@ -91,32 +79,30 @@ You can change configurations in `conf/master.ini` according to cluster informat
hostName=YOUR_SERVER_IP // replaced with your server IP
port=8099
webPort=8080
+metaDataPath=/stage/meta_data
```
+
- Access Authorization Token
```ini
confModAuthToken=abc // for configuring Web Resources\API etc
```
+
- ZooKeeper Cluster
```ini
-[zookeeper]
+[zookeeper] // Master and Broker in the same cluster must use the same zookeeper environment and have the same configuration
zkNodeRoot=/tubemq
zkServerAddr=localhost:2181 // multi zookeeper addresses can separate with ","
```
+
- Replication Strategy
```ini
[replication]
-repNodeName=tubemqMasterGroupNode1 // using different name for each master node
+repGroupName=tubemqGroup1 // The Master of the same cluster must use the same group name, and the group names of different clusters must be different
+repNodeName=tubemqGroupNode1 // The master node names of the same cluster must be different names
repHelperHost=FIRST_MASTER_NODE_IP:9001 // helperHost is used for building HA master.
```
-- Resource Path of Web
-
-You can change configurations in `resources/velocity.properties`
-```properties
-file.resource.loader.path=/INSTALL_PATH/apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin/resources/templates
-```
- (Optional) Master High Availability
-
In the example above, we run the services on a single node. However, in real production environment, you
need to run multiple master services on different servers for high availability purpose. Here's
the introduction of availability level.
@@ -129,7 +115,8 @@ the introduction of availability level.
**Tips**：Please notice that the master servers should be clock synchronized.
-#### Configure Broker
+
+### Configure Broker
You can change configurations in `conf/broker.ini` according to cluster information.
- Broker IP and Port
```ini
@@ -143,33 +130,33 @@ webPort=8081
```ini
masterAddressList=MASTER_NODE_IP1:8099,MASTER_NODE_IP2:8099 // multi addresses can separate with ","
```
+
- Metadata Path
```ini
primaryPath=/stage/msg_data
```
+
- ZooKeeper Cluster
```ini
-[zookeeper]
+[zookeeper] // Master and Broker in the same cluster must use the same zookeeper environment and have the same configuration
zkNodeRoot=/tubemq
zkServerAddr=localhost:2181 // multi zookeeper addresses can separate with ","
```
-#### Start Master
+### Start Master
Please go to the `bin` folder and run this command to start
the master service.
```bash
-./tubemq master start
+./tubemq.sh master start
```
+
You should be able to access `http://your-master-ip:8080` to see the
web GUI now.
#### Configure Broker Metadata
-Before we start a broker service, we need to configure it on master web GUI first.
-
-Go to the `Broker List` page, click `Add Single Broker`, and input the new broker
-information.
+Before we start a broker service, we need to configure it on master web GUI first. Go to the `Broker List` page, click `Add Single Broker`, and input the new broker information.
@@ -182,10 +169,10 @@ Click the online link to activate the new added broker.
-#### Start Broker
+### Start Broker
Please go to the `bin` folder and run this command to start the broker service
```bash
-./tubemq broker start
+./tubemq.sh broker start
```
Refresh the GUI broker list page, you can see that the broker now is registered.
@@ -247,6 +234,9 @@ cd /INSTALL_PATH/apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin
From the log, we can see the message received by the consumer.
+## The End
+Here, the compilation, deployment, system configuration, startup, production and consumption of TubeMQ have been completed. If you need to understand more in-depth content, please check the relevant content in "TubeMQ HTTP API" and make the corresponding configuration settings.
+
---
diff --git a/docs/en-us/quick_start.md b/docs/en-us/quick_start.md
index 8bfa744..2b8b414 100644
--- a/docs/en-us/quick_start.md
+++ b/docs/en-us/quick_start.md
@@ -18,8 +18,8 @@ Through the overall architecture diagram of InLong above, we can see that InLong
## Compile and deploy Manager
[How to compile and deploy Manager](modules/manager/quick_start.md)
-## Compile and deploy Manager web
-[How to compile and deploy Manager Web](modules/console/quick_start.md)
+## Compile and deploy InLong Web
+[How to compile and deploy InLong Web](modules/console/quick_start.md)
## Compile and deploy TubeMQ
[How to compile and deploy TubeMQ](modules/tubemq/quick_start.md)
diff --git a/docs/zh-cn/modules/tubemq/quick_start.md b/docs/zh-cn/modules/tubemq/quick_start.md
index 5b0acf2..aa85c5b 100644
--- a/docs/zh-cn/modules/tubemq/quick_start.md
+++ b/docs/zh-cn/modules/tubemq/quick_start.md
@@ -3,8 +3,9 @@ title: 快速开始 - Apache InLong TubeMQ模块
---
## 编译和构建
+
### 准备工作
-- Java JDK 1.7 或 1.8
+- Java JDK 1.8
- Maven 3.3+
### 从源码包构建
@@ -12,22 +13,22 @@ title: 快速开始 - Apache InLong TubeMQ模块
```bash
mvn clean package -DskipTests
```
-- （可选）使用Docker编译：
-```bash
-docker run -v REPLACE_WITH_SOURCE_PATH:/tubemq apachetubemq/tubemq-build clean package -DskipTests
-```
+
- 单元测试：
```bash
mvn test
```
+
- 单独对每个 module 进行构建：
```bash
mvn clean install
cd module-name (比如: tubemq-client)
mvn test
```
+
构建完成之后，在 `tubemq-server/target` 目录下会有 **apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin.tar.gz** 文件。
-这是 TubeMq 的部署包，包含了脚本、配置文件、依赖以及 web GUI相关的内容。
+这是 TubeMQ 的部署包，包含了脚本、配置文件、依赖以及 web GUI相关的内容。
+
### 配置IDE开发环境
在IDE中构建和调试源码，需要先运行以下命令：
```bash
@@ -42,18 +43,18 @@ mvn compile
<protocExecutable>/usr/local/bin/protoc</protocExecutable>
</configuration>
```
+
## 部署运行
-### 部署TubeMQ Standalone
-Standalone模式可以在一个容器中启动zookeeper/master/broker服务：
-```
-docker run -p 8080:8080 -p 8000:8000 -p 8123:8123 --name tubemq -d apachetubemq/tubemq-all:latest
-```
-容器拉起后，可在浏览器访问` http://127.0.0.1:8080`，然后参考下面`快速使用`部分开始使用。
-**注意**：Standalone模式只可用于开发和体验，不可作为生产集群。
+### 配置示例
+TubeMQ 集群包含有两个组件: **Master** 和 **Broker**. Master 和 Broker 可以部署在相同或者不同的节点上，依照业务对机器的规划进行处理。我们通过如下3台机器搭建有2台Master的生产、消费的集群进行配置示例：
+| 所属角色 | TCP端口 | TLS端口 | WEB端口 | 备注 |
+| --- | --- | --- | --- | --- |
+| Master | 8099 | 8199 | 8080 | 元数据存储在`/stage/meta_data` |
+| Broker | 8123 | 8124 | 8081 | 消息储存在`/stage/msg_data` |
+| ZooKeeper | 2181 | | | Offset储存在根目录`/tubemq` |
-### 部署TubeMQ集群
-#### 准备工作
+### 准备工作
- ZooKeeper集群
- [apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin.tar.gz](download/download.md)安装包
@@ -66,49 +67,40 @@ docker run -p 8080:8080 -p 8000:8000 -p 8123:8123 --name tubemq -d apachetubemq/
├── logs
└── resources
```
-#### 配置示例
-TubeMQ 集群包含有两个组件: **Master** 和 **Broker**. Master 和 Broker 可以部署在相同或者不同的节点上。下面是
-一个集群的配置示例：
-
-| Role | TCP Port | TLS Port | Web Port | Comment |
-| ---- | -------- | -------- | -------- | ------- |
-| Master | 8099 | 8199 | 8080 | 元数据存储在 /stage/meta_data |
-| Broker | 8123 | 8124 | 8081 | 消息存储在 /stage/msg_data |
-| Zookeeper | 2181 | | | Offset 存储在 /tubemq |
-#### 配置Master
+### 配置Master
编辑`conf/master.ini`，根据集群信息变更以下配置项
+
- Master IP和端口
```ini
[master]
-hostName=YOUR_SERVER_IP // 替换为当前主机IP
+hostName=YOUR_SERVER_IP // 替换为当前主机IP
port=8099
webPort=8080
+metaDataPath=/stage/meta_data
```
+
- 访问授权Token
```ini
-confModAuthToken=abc // 该token用于页面配置、API调用等
+confModAuthToken=abc // 该token用于页面配置、API调用等
```
+
- ZooKeeper集群地址
```ini
-[zookeeper]
+[zookeeper] // 同一个集群里Master和Broker必须使用同一套zookeeper环境，且配置一致
zkNodeRoot=/tubemq
-zkServerAddr=localhost:2181 // 指向zookeeper集群，多个地址逗号分开
+zkServerAddr=localhost:2181 // 指向zookeeper集群，多个地址逗号分开
```
+
- 配置Replication策略
```ini
[replication]
-repNodeName=tubemqMasterGroupNode1 // 每个master节点需使用不同名称
+repGroupName=tubemqGroup1 // 同一个集群的Master必须要用同一个组名，且不同集群的组名必须不同
+repNodeName=tubemqGroupNode1 // 同一个集群的master节点名必须是不同的名称
repHelperHost=FIRST_MASTER_NODE_IP:9001 // helperHost用于创建master集群，一般配置第一个master节点ip
```
-- 前端安装路径
-编辑resources/velocity.properties
-```properties
-file.resource.loader.path=/INSTALL_PATH/apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin/resources/templates
-```
- （可选）生产环境，多master HA级别
-
| HA级别 | Master数量 | 描述 |
| -------- | ------------- | ----------- |
| 高 | 3 masters | 任何主节点崩溃后，集群元数据仍处于读/写状态，可以接受新的生产者/消费者。 |
@@ -117,7 +109,8 @@ file.resource.loader.path=/INSTALL_PATH/apache-inlong-tubemq-server-[TUBEMQ-VERS
**注意**：需保证Master所有节点之间的时钟同步
-#### 配置Broker
+
+### 配置Broker
编辑`conf/broker.ini`，根据集群信息变更以下配置项
- Broker IP和端口
```ini
@@ -127,30 +120,33 @@ hostName=YOUR_SERVER_IP // 替换为当前主机IP，broker目
port=8123
webPort=8081
```
+
- Master地址
```ini
masterAddressList=YOUR_MASTER_IP1:8099,YOUR_MASTER_IP2:8099 //多个master以逗号分隔
```
+
- 数据目录
```ini
primaryPath=/stage/msg_data
```
+
- ZooKeeper集群地址
```ini
-[zookeeper]
-zkNodeRoot=/tubemq
+[zookeeper] // 同一个集群里Master和Broker必须使用同一套zookeeper环境，且配置一致
+zkNodeRoot=/tubemq
zkServerAddr=localhost:2181 // 指向zookeeper集群，多个地址逗号分开
```
-#### 启动Master
+### 启动Master
进入Master节点的 `bin` 目录下，启动服务:
```bash
-./tubemq master start
+./tubemq.sh master start
```
访问Master的管控台 `http://YOUR_MASTER_IP:8080` ，页面可查则表示master已成功启动:
-
+
#### 配置Broker元数据
Broker启动前，首先要在Master上配置Broker元数据，增加Broker相关的管理信息。在`Broker List` 页面, `Add Single Broker`，然后填写相关信息:
@@ -161,37 +157,31 @@ Broker启动前，首先要在Master上配置Broker元数据，增加Broker相
2. authToken: `conf/master.ini` 文件中 `confModAuthToken` 字段配置的 token
然后上线Broker：
-
-#### 启动Broker
+### 启动Broker
进入broker节点的 `bin` 目录下，执行以下命令启动Broker服务：
```bash
-./tubemq broker start
+./tubemq.sh broker start
```
-刷新页面可以看到 Broker 已经注册，当 `当前运行子状态` 为 `idle` 时， 可以增加topic。
-
+刷新页面可以看到 Broker 已经注册，当 `当前运行子状态` 为 `idle` 时， 可以增加topic:
## 快速使用
### 新增 Topic
-可以通过 web GUI 添加 Topic， 在 `Topic列表`页面添加，需要填写相关信息，比如增加`demo` topic：
+可以通过 web GUI 添加 Topic， 在 `Topic列表`页面添加，需要填写相关信息，比如增加`demo` topic：
然后选择部署 Topic 的 Broker
-
此时 Broker的 `可发布` 和 `可订阅` 依旧是灰色的
-
-
需要在 `Broker列表`页面重载Broker 配置
-
@@ -205,9 +195,7 @@ Broker启动前，首先要在Master上配置Broker元数据，增加Broker相
可以通过上面创建的`demo` topic来测试集群。
- 生产消息
-
将 `YOUR_MASTER_IP:port` 替换为实际的IP和端口，然后运行producer:
-
```bash
cd /INSTALL_PATH/apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin
./bin/tubemq-producer-test.sh --master-servers YOUR_MASTER_IP1:port,YOUR_MASTER_IP2:port --topicName demo
@@ -217,7 +205,6 @@ cd /INSTALL_PATH/apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin
- 消费消息
-
将 `YOUR_MASTER_IP:port` 替换为实际的IP和端口，然后运行Consumer:
```bash
cd /INSTALL_PATH/apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin
@@ -225,9 +212,13 @@ cd /INSTALL_PATH/apache-inlong-tubemq-server-[TUBEMQ-VERSION]-bin
```
如果能观察下如下日志，则表示数据被消费者消费到：
+
+## 结束
+在这里，已经完成了TubeMQ的编译，部署，系统配置，启动，生产和消费。如果需要了解更深入的内容，请查看《TubeMQ HTTP API》里的相关内容，进行相应的配置设置。
+
---
diff --git a/docs/zh-cn/quick_start.md b/docs/zh-cn/quick_start.md
index 10f5aa5..9dd5ff3 100644
--- a/docs/zh-cn/quick_start.md
+++ b/docs/zh-cn/quick_start.md
@@ -18,8 +18,8 @@ InLong(应龙)最初建于腾讯，服务线上业务8年多，支持大数据
## 编译部署Manager
[如何编译部署Manager](modules/manager/quick_start.md)
-## 编译部署Manager的web
-[如何编译部署Manager Web](modules/console/quick_start.md)
+## 编译部署InLong Web
+[如何编译部署InLong Web](modules/console/quick_start.md)
## 编译部署TubeMQ
[如何编译部署TubeMQ](modules/tubemq/quick_start.md)
diff --git a/site_config/docs.js b/site_config/docs.js
index 40f6f21..f7a00aa 100644
--- a/site_config/docs.js
+++ b/site_config/docs.js
@@ -64,8 +64,8 @@ export default {
link: '/en-us/docs/modules/tubemq/architecture.html',
},
{
- title: 'Deployment',
- link: '/en-us/docs/modules/tubemq/deployment.html',
+ title: 'Quick Start',
+ link: '/en-us/docs/modules/tubemq/quick_start.html',
},
{
title: 'Configure Introduction',
@@ -84,6 +84,10 @@ export default {
link: '/en-us/docs/modules/tubemq/error_code.html',
},
{
+ title: 'Deployment Demo',
+ link: '/en-us/docs/modules/tubemq/deployment.html',
+ },
+ {
title: 'HTTP API',
link: '/en-us/docs/modules/tubemq/http_access_api.html',
},
@@ -98,7 +102,7 @@ export default {
children: [
{
title: 'Build && Deployment',
- link: '/en-us/docs/modules/tubemq-manager/quick_start.html',
+ link: '/en-us/docs/modules/tubemq/tubemq-manager/quick_start.html',
},
],
},
@@ -193,8 +197,8 @@ export default {
link: '/zh-cn/docs/modules/tubemq/architecture.html',
},
{
- title: '部署指引',
- link: '/zh-cn/docs/modules/tubemq/deployment.html',
+ title: '快速开始',
+ link: '/zh-cn/docs/modules/tubemq/quick_start.html',
},
{
title: '配置参数介绍',
@@ -213,6 +217,10 @@ export default {
link: '/zh-cn/docs/modules/tubemq/error_code.html',
},
{
+ title: '部署示例',
+ link: '/zh-cn/docs/modules/tubemq/deployment.html',
+ },
+ {
title: 'HTTP API介绍',
link: '/zh-cn/docs/modules/tubemq/http_access_api.html',
},
@@ -227,7 +235,7 @@ export default {
children: [
{
title: '编译部署',
- link: '/zh-cn/docs/modules/tubemq-manager/quick_start.html',
+ link: '/zh-cn/docs/modules/tubemq/tubemq-manager/quick_start.html',
},
],
},