You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@inlong.apache.org by go...@apache.org on 2021/07/07 14:49:03 UTC
[incubator-inlong-website] branch master updated: [INLONG-679]
adjust start.md in agent proxy and tubemanager (#86)
This is an automated email from the ASF dual-hosted git repository.
gosonzhang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-inlong-website.git
The following commit(s) were added to refs/heads/master by this push:
new 533d37a [INLONG-679] adjust start.md in agent proxy and tubemanager (#86)
533d37a is described below
commit 533d37afa3830521f2a05f6d0109a0c1fb936823
Author: ziruipeng <zp...@connect.ust.hk>
AuthorDate: Wed Jul 7 22:48:55 2021 +0800
[INLONG-679] adjust start.md in agent proxy and tubemanager (#86)
Co-authored-by: stingpeng <st...@tencent.com>
---
docs/en-us/modules/agent/quick_start.md | 3 +-
docs/en-us/modules/dataproxy/quick_start.md | 25 ++++++++++-----
.../modules/tubemq/tubemq-manager/quick_start.md | 33 +++++++++----------
docs/zh-cn/modules/agent/quick_start.md | 7 ++--
docs/zh-cn/modules/dataproxy/quick_start.md | 30 ++++++++++++------
.../modules/tubemq/tubemq-manager/quick_start.md | 37 +++++++++++-----------
6 files changed, 77 insertions(+), 58 deletions(-)
diff --git a/docs/en-us/modules/agent/quick_start.md b/docs/en-us/modules/agent/quick_start.md
index 985c249..f2a2840 100644
--- a/docs/en-us/modules/agent/quick_start.md
+++ b/docs/en-us/modules/agent/quick_start.md
@@ -29,7 +29,7 @@ Create a new folder named .inlong\.managerIps in the agent directory, and create
write:
- {"cluster_id":45,"isInterVisit":1,"size":1,"address": [{"port":write proxy port,"host":"write proxy ip"}], "switch":0}
+ {"cluster_id":1,"isInterVisit":1,"size":1,"address": [{"port":write proxy port,"host":"write proxy ip"}], "switch":0}
###2.3 local job configuration
If you do not use the online mode, you can use the local file to create a new read task
@@ -37,7 +37,6 @@ Create a new directory jobs in conf,
vim job1.json：
put
-{
"job": {
diff --git a/docs/en-us/modules/dataproxy/quick_start.md b/docs/en-us/modules/dataproxy/quick_start.md
index f98748f..d556fbf 100644
--- a/docs/en-us/modules/dataproxy/quick_start.md
+++ b/docs/en-us/modules/dataproxy/quick_start.md
@@ -2,10 +2,11 @@
mvn clean package -DskipTests
-The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz can be found under the inlong-dataproxy/dataproxy-dist/target project
+The tgz installation package apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz can be found under the inlong-dataproxy/dataproxy-dist/target
- un tar it
- tar -zxvf apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz
+un tar it
+
+ tar -zxvf apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz
## 2、configuration
@@ -13,9 +14,17 @@ The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.t
cd apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin
### config tube
- vim conf/flume.conf
- In vim command line mode, execute the following replacement commands, where ip:port is the tube address
- :%s/TUBE_LIST/ip:port,ip:port
+
+ ip:port are tube address
+
+sed -i 's/TUBE_LIST/ip1:port,ip2:port/g' conf/flume.conf
+
+ change proxy port
+ agent1.sources.tcp-source.port = 46801
+
+notice that conf/flume.conf FLUME_HOME is proxy the directory for proxy inner data
+
+
### Environmental preparation
cd bin
@@ -23,10 +32,10 @@ The tgz installation package apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.t
cd ..
### config manager
- vim conf/commmon.properties
+ vim conf/common.properties
write：
cluster_id=1
- manager_hosts=xx.xx.xx.xx:port
+ manager_hosts=ip:port
## 3、run
sh bin/start.sh
diff --git a/docs/en-us/modules/tubemq/tubemq-manager/quick_start.md b/docs/en-us/modules/tubemq/tubemq-manager/quick_start.md
index e893718..a2f8d3d 100644
--- a/docs/en-us/modules/tubemq/tubemq-manager/quick_start.md
+++ b/docs/en-us/modules/tubemq/tubemq-manager/quick_start.md
@@ -8,40 +8,42 @@ environment requirements：
mysql
java(1.8+)
-In the dist directory, you can find the file tubemq-manager-bin.zip
+In the InLong/inlong-tubemq/tubemq-manager/dist/ directory, you can find the file apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
- unzip tubemq-manager-bin.zip
+ tar -zxvf apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
Add mysql information in conf/application.properties:
spring.jpa.hibernate.ddl-auto=update
# configuration for manager
- spring.datasource.url=jdbc:mysql://x.x.x.x:3306/tubemanager
- spring.datasource.username=xx
- spring.datasource.password=xxx
+ spring.datasource.url=jdbc:mysql://[replace_by_mysql_address]:3306/tubemanager
+ spring.datasource.username=[replace_by_usename]
+ spring.datasource.password=[replace_by_password]
+ # server port
+ server.port=8089
Add the database tubemanager and start:
- bin/start-manager.sh
+ bin/start-admin.sh
## use
### create cluster
Before using tubeAdmin to operate the cluster, you first need to register the cluster information, and use the following interface to add a cluster:
- /v1/cluster?method=add
+/v1/cluster?method=add
POST
parameter:
{
- "masterIp": "9.23.28.86", (tube master ip)
+ "masterIp": "127.0.0.1", (tubemq master ip)
"clusterName": "test",
- "masterPort": "8000", (tube master port)
- "masterWebPort": "8080", (tube master web port)
+ "masterPort": "8000", (tubemq master port)
+ "masterWebPort": "8080", (tubemq master web port)
"createUser": "test",
- "token": "abc" (tube token)
+ "token": "abc" (tubemq token)
}
### Operation interface
@@ -82,10 +84,10 @@ AddTopicTasks is a list of the following objects, which can carry multiple creat
topicName (required) topic name
-Example
+Example
POST
- /v1/task?method=addTopicTask
+/v1/task?method=addTopicTask
{
"clusterId": "1",
@@ -115,10 +117,9 @@ example
POST
- /v1/topic?method=queryCanWrite
{
- "clusterId": "2",
- "topicName": "pzrTestss1123123",
+ "clusterId": "1",
+ "topicName": "1",
"user": "test"
}
diff --git a/docs/zh-cn/modules/agent/quick_start.md b/docs/zh-cn/modules/agent/quick_start.md
index c387c0f..2bfe2ce 100644
--- a/docs/zh-cn/modules/agent/quick_start.md
+++ b/docs/zh-cn/modules/agent/quick_start.md
@@ -1,4 +1,4 @@
-##一、安装
+## ~~~~一、安装
环境要求：
java JDK 1.8
@@ -11,7 +11,6 @@
-
## 二、配置
agent 支持两种运行模式：本地运行以及线上运行
@@ -29,7 +28,7 @@ agent.manager.vip.http.port=manager port
内部写入：
- {"cluster_id":45,"isInterVisit":1,"size":1,"address": [{"port":写入proxy port,"host":"写入proxy ip"}], "switch":0}
+ {"cluster_id":1,"isInterVisit":1,"size":1,"address": [{"port":写入proxy port,"host":"写入proxy ip"}], "switch":0}
###2.3 agent本地运行job配置
如果不使用线上模式，可以使用本地文件新建读取任务
@@ -44,7 +43,7 @@ vim job1.json：
"dir": {
- "pattern": "需要读取的文件路径，可包含正则表达式"
+ "pattern": "需要读取的文件路径，可包含正则表达式，例如test/inlong_[0-9]{1}，监听test下创建的新文件"
},
diff --git a/docs/zh-cn/modules/dataproxy/quick_start.md b/docs/zh-cn/modules/dataproxy/quick_start.md
index 337ab40..07b0882 100644
--- a/docs/zh-cn/modules/dataproxy/quick_start.md
+++ b/docs/zh-cn/modules/dataproxy/quick_start.md
@@ -2,20 +2,29 @@
mvn clean package -DskipTests
-在inlong-dataproxy/dataproxy-dist/target项目下可以找到tgz安装包apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz
+在inlong-dataproxy/dataproxy-dist/target项目下可以找到tgz安装包apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz
- 解压
- tar -zxvf apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz
+
+解压
+
+ tar -zxvf apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin.tar.gz
## 二、配置
- cd apache-inlong-proxy-0.9.0-incubating-SNAPSHOT-bin
+ cd apache-inlong-dataproxy-0.9.0-incubating-SNAPSHOT-bin
+
+### 配置tube地址和端口号
+
+ 执行以下替换命令,其中ip:port为tube地址
-### 配置tube地址
- vim conf/flume.conf
- vim命令行模式，执行以下替换命令,其中ip:port为tube master地址，可配置多个
- :%s/TUBE_LIST/ip1:port,ip2:port
+sed -i 's/TUBE_LIST/ip1:port,ip2:port/g' conf/flume.conf
+
+
+ 修改以下内容为proxy端口号
+ agent1.sources.tcp-source.port = 46801
+
+注意conf/flume.conf中FLUME_HOME为proxy的中间数据文件存放地址
### 环境准备
cd bin
@@ -23,10 +32,11 @@
cd ..
### 配置manager地址
- vim conf/commmon.properties
+ vim conf/common.properties
+
写入以下配置：
cluster_id=1
- manager_hosts=xx.xx.xx.xx:port
+ manager_hosts=ip:port // 这里写入的是manager_api地址
## 三、启动
sh bin/start.sh
diff --git a/docs/zh-cn/modules/tubemq/tubemq-manager/quick_start.md b/docs/zh-cn/modules/tubemq/tubemq-manager/quick_start.md
index 9a8f3e5..f060931 100644
--- a/docs/zh-cn/modules/tubemq/tubemq-manager/quick_start.md
+++ b/docs/zh-cn/modules/tubemq/tubemq-manager/quick_start.md
@@ -8,21 +8,23 @@
mysql
java(1.8+)
-在dist目录当中，能够找到文件为tubemq-manager-bin.zip
+在InLong/inlong-tubemq/tubemq-manager/dist/目录当中，能够找到文件为apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
- unzip tubemq-manager-bin.zip
+ tar -zxvf apache-inlong-tubemanager-0.9.0-incubating-SNAPSHOT-bin.tar.gz
在conf/application.properties中添加mysql信息：
spring.jpa.hibernate.ddl-auto=update
# configuration for manager
- spring.datasource.url=jdbc:mysql://x.x.x.x:3306/tubemanager
- spring.datasource.username=xx
- spring.datasource.password=xxx
+ spring.datasource.url=jdbc:mysql://[replace_by_mysql_address]:3306/tubemanager
+ spring.datasource.username=[replace_by_usename]
+ spring.datasource.password=[replace_by_password]
+ # server port
+ server.port=8089
添加数据库tubemanager，并启动：
- bin/start-manager.sh
+ bin/start-admin.sh
## 使用
@@ -37,12 +39,12 @@ java(1.8+)
参数：
{
- "masterIp": "9.23.28.86", (tube master ip)
+ "masterIp": "127.0.0.1", (tubemq master ip)
"clusterName": "test",
- "masterPort": "8000", (tube master port)
- "masterWebPort": "8080", (tube master web port)
+ "masterPort": "8000", (tubemq master port)
+ "masterWebPort": "8080", (tubemq master web port)
"createUser": "test",
- "token": "abc" (tube token)
+ "token": "abc" (tubemq token)
}
### 操作接口
@@ -51,8 +53,8 @@ java(1.8+)
查询clusterId以及clusterName全量数据 （get)
示例
- GET
- /v1/cluster
+GET
+/v1/cluster
返回值
@@ -65,7 +67,7 @@ java(1.8+)
#### topic
-#####添加topicTask
+#### 添加topicTask
type (必填) 请求类型，字段填写：op_query
clusterId (必填) 请求集群id
@@ -84,7 +86,7 @@ AddTopicTasks 为以下对象的List，可携带多个创建topic请求
示例
POST
- /v1/task?method=addTopicTask
+/v1/task?method=addTopicTask
{
"clusterId": "1",
@@ -104,8 +106,7 @@ POST
result为false为写入task失败
-
-#####查询某一个topic是否创建成功（业务可以写入）
+#### 查询某一个topic是否创建成功（业务可以写入）
clusterId (必填) 请求集群id
topicName (必填) 查询topic名称
@@ -119,8 +120,8 @@ POST
/v1/topic?method=queryCanWrite
{
- "clusterId": "2",
- "topicName": "pzrTestss1123123",
+ "clusterId": "1",
+ "topicName": "1",
"user": "test"
}