You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/02/22 12:41:06 UTC
[GitHub] [apisix] juzhiyuan commented on a change in pull request #3615: feat: add consul kv discovery module
juzhiyuan commented on a change in pull request #3615:
URL: https://github.com/apache/apisix/pull/3615#discussion_r580217831
##########
File path: doc/discovery.md
##########
@@ -243,3 +244,8 @@ $ curl http://127.0.0.1:9080/apisix/admin/routes/2 -H 'X-API-KEY: edd1c9f034335f
Suppose both A-SERVICE and B-SERVICE provide a `/test` API. The above configuration allows access to A-SERVICE's `/test` API through `/a/test` and B-SERVICE's `/test` API through `/b/test`.
**Notice**：When configuring `upstream.service_name`, `upstream.nodes` will no longer take effect, but will be replaced by 'nodes' obtained from the registry.
+
+## Discovery modules
+
+- eureka
+- [consul key&value](discovery/consul_kv.md)
Review comment:
```suggestion
- [Consul KV](discovery/consul_kv.md)
```
##########
File path: doc/discovery/consul_kv.md
##########
@@ -0,0 +1,148 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+<!--[Chinese](zh-cn/discovery.md)-->`
+
+## Summary
+
+For users who used [nginx-upsync-module](https://github.com/weibocom/nginx-upsync-module) and consul key value for service discovery way, as we Weibo Mobile Team, maybe need it.
+
+Thanks to @fatman-x guy, who developed this module, called `consul_kv`, and its worker process data flow is below:
+
+
+## Configuration for discovery client
+
+### Configuration for Consul KV
+
+Add following configuration in `conf/config.yaml` :
+
+```yaml
+discovery:
+ consul_kv:
+ servers:
+ - "http://127.0.0.1:8500"
+ - "http://127.0.0.1:8600"
+ prefix: "upstreams"
+ skip_keys: # if you need to skip special keys
+ - "upstreams/unused_api/"
+ timeout:
+ connect: 1000 # default 2000 ms
+ read: 1000 # default 2000 ms
+ wait: 60 # default 60 sec
+ weight: 1 # default 1
+ fetch_interval: 5 # default 3 sec, only take effect for keepalive: short way
+ keepalive: true # default true, use the long pull way to query consul servers
+ default_server: # you can define default server when missing hit
+ host: "127.0.0.1"
+ port: 20999
+ metadata:
+ fail_timeout: 1 # default 1 ms
+ weight: 1 # default 1
+ max_fails: 1 # default 1
+```
+
+And you can config it in short by default value:
+
+```yaml
+discovery:
+ consul_kv:
+ servers:
+ - "http://127.0.0.1:8500"
+```
+
+The `keepalive` has two optional values:
+
+- `true`, default and recommend value, use the long pull way to query consul servers
+- `false`, not recommend, it would use the short pull way to query consul servers, then you can set the `fetch_interval` for fetch interval
+
+### Register Http API Services
+
+Service register Key&Value template:
+
+```
+Key: {Prefix}/{Service_Name}/{IP}:{Port}
+Value: {"weight": <Num>, "max_fails": <Num>, "fail_timeout": <Num>}
+```
+
+The register consul key use `upstreams` as prefix by default. The http api service name called `webpages` for example, and you can also use `webpages/oneteam/hello` as service name. The api instance of node's ip and port make up new key: `<IP>:<Port>`.
+
+Now, register nodes into consul:
+
+```bash
Review comment:
How about using `shell` here just like L105
##########
File path: doc/discovery/consul_kv.md
##########
@@ -0,0 +1,148 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+<!--[Chinese](zh-cn/discovery.md)-->`
+
+## Summary
+
+For users who used [nginx-upsync-module](https://github.com/weibocom/nginx-upsync-module) and consul key value for service discovery way, as we Weibo Mobile Team, maybe need it.
+
+Thanks to @fatman-x guy, who developed this module, called `consul_kv`, and its worker process data flow is below:
+
+
+## Configuration for discovery client
+
+### Configuration for Consul KV
+
+Add following configuration in `conf/config.yaml` :
+
+```yaml
+discovery:
+ consul_kv:
+ servers:
+ - "http://127.0.0.1:8500"
+ - "http://127.0.0.1:8600"
+ prefix: "upstreams"
+ skip_keys: # if you need to skip special keys
+ - "upstreams/unused_api/"
+ timeout:
+ connect: 1000 # default 2000 ms
+ read: 1000 # default 2000 ms
+ wait: 60 # default 60 sec
+ weight: 1 # default 1
+ fetch_interval: 5 # default 3 sec, only take effect for keepalive: short way
+ keepalive: true # default true, use the long pull way to query consul servers
+ default_server: # you can define default server when missing hit
+ host: "127.0.0.1"
+ port: 20999
+ metadata:
+ fail_timeout: 1 # default 1 ms
+ weight: 1 # default 1
+ max_fails: 1 # default 1
+```
+
+And you can config it in short by default value:
+
+```yaml
+discovery:
+ consul_kv:
+ servers:
+ - "http://127.0.0.1:8500"
+```
+
+The `keepalive` has two optional values:
+
+- `true`, default and recommend value, use the long pull way to query consul servers
+- `false`, not recommend, it would use the short pull way to query consul servers, then you can set the `fetch_interval` for fetch interval
+
+### Register Http API Services
+
+Service register Key&Value template:
+
+```
+Key: {Prefix}/{Service_Name}/{IP}:{Port}
+Value: {"weight": <Num>, "max_fails": <Num>, "fail_timeout": <Num>}
+```
+
+The register consul key use `upstreams` as prefix by default. The http api service name called `webpages` for example, and you can also use `webpages/oneteam/hello` as service name. The api instance of node's ip and port make up new key: `<IP>:<Port>`.
+
+Now, register nodes into consul:
+
+```bash
+curl \
+ -X PUT \
+ -d ' {"weight": 1, "max_fails": 2, "fail_timeout": 1}' \
+ http://127.0.0.1:8500/v1/kv/upstreams/webpages/172.19.5.12:8000
+
+curl \
+ -X PUT \
+ -d ' {"weight": 1, "max_fails": 2, "fail_timeout": 1}' \
+ http://127.0.0.1:8500/v1/kv/upstreams/webpages/172.19.5.13:8000
+```
+
+In some case, same keys exist in different consul servers.
+To avoid confusion, use the full consul key url path as service name in practice.
+
+### Upstream setting
+
+Here is an example of routing a request with a URL of "/*" to a service which named "http://127.0.0.1:8500/v1/kv/upstreams/webpages/" and use consul_kv discovery client in the registry :
+
+```shell
+$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -i -d '
+{
+ "uri": "/*",
+ "upstream": {
+ "service_name": "http://127.0.0.1:8500/v1/kv/upstreams/webpages/",
+ "type": "roundrobin",
+ "discovery_type": "consul_kv"
+ }
+}'
+```
+
+The format response as below:
+
+```json
+{
+ "node": {
+ "value": {
+ "priority": 0,
+ "update_time": 1612755230,
+ "upstream": {
+ "discovery_type": "consul_kv",
+ "service_name": "http://127.0.0.1:8500/v1/kv/upstreams/webpages/",
+ "hash_on": "vars",
+ "type": "roundrobin",
+ "pass_host": "pass"
+ },
+ "id": "1",
+ "uri": "/*",
+ "create_time": 1612755230,
+ "status": 1,
+ "saas_id": ""
Review comment:
Do we need this field?
##########
File path: doc/discovery/consul_kv.md
##########
@@ -0,0 +1,148 @@
+<!--
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+-->
+<!--[Chinese](zh-cn/discovery.md)-->`
+
+## Summary
+
+For users who used [nginx-upsync-module](https://github.com/weibocom/nginx-upsync-module) and consul key value for service discovery way, as we Weibo Mobile Team, maybe need it.
+
+Thanks to @fatman-x guy, who developed this module, called `consul_kv`, and its worker process data flow is below:
+
+
+## Configuration for discovery client
+
+### Configuration for Consul KV
+
+Add following configuration in `conf/config.yaml` :
+
+```yaml
+discovery:
+ consul_kv:
+ servers:
+ - "http://127.0.0.1:8500"
+ - "http://127.0.0.1:8600"
+ prefix: "upstreams"
+ skip_keys: # if you need to skip special keys
+ - "upstreams/unused_api/"
+ timeout:
+ connect: 1000 # default 2000 ms
+ read: 1000 # default 2000 ms
+ wait: 60 # default 60 sec
+ weight: 1 # default 1
+ fetch_interval: 5 # default 3 sec, only take effect for keepalive: short way
+ keepalive: true # default true, use the long pull way to query consul servers
+ default_server: # you can define default server when missing hit
+ host: "127.0.0.1"
+ port: 20999
+ metadata:
+ fail_timeout: 1 # default 1 ms
+ weight: 1 # default 1
+ max_fails: 1 # default 1
+```
+
+And you can config it in short by default value:
+
+```yaml
+discovery:
+ consul_kv:
+ servers:
+ - "http://127.0.0.1:8500"
+```
+
+The `keepalive` has two optional values:
+
+- `true`, default and recommend value, use the long pull way to query consul servers
+- `false`, not recommend, it would use the short pull way to query consul servers, then you can set the `fetch_interval` for fetch interval
+
+### Register Http API Services
+
+Service register Key&Value template:
+
+```
+Key: {Prefix}/{Service_Name}/{IP}:{Port}
+Value: {"weight": <Num>, "max_fails": <Num>, "fail_timeout": <Num>}
+```
+
+The register consul key use `upstreams` as prefix by default. The http api service name called `webpages` for example, and you can also use `webpages/oneteam/hello` as service name. The api instance of node's ip and port make up new key: `<IP>:<Port>`.
+
+Now, register nodes into consul:
+
+```bash
+curl \
+ -X PUT \
+ -d ' {"weight": 1, "max_fails": 2, "fail_timeout": 1}' \
+ http://127.0.0.1:8500/v1/kv/upstreams/webpages/172.19.5.12:8000
+
+curl \
+ -X PUT \
+ -d ' {"weight": 1, "max_fails": 2, "fail_timeout": 1}' \
+ http://127.0.0.1:8500/v1/kv/upstreams/webpages/172.19.5.13:8000
+```
+
+In some case, same keys exist in different consul servers.
+To avoid confusion, use the full consul key url path as service name in practice.
+
+### Upstream setting
+
+Here is an example of routing a request with a URL of "/*" to a service which named "http://127.0.0.1:8500/v1/kv/upstreams/webpages/" and use consul_kv discovery client in the registry :
+
+```shell
+$ curl http://127.0.0.1:9080/apisix/admin/routes/1 -H 'X-API-KEY: edd1c9f034335f136f87ad84b625c8f1' -X PUT -i -d '
+{
+ "uri": "/*",
+ "upstream": {
+ "service_name": "http://127.0.0.1:8500/v1/kv/upstreams/webpages/",
+ "type": "roundrobin",
+ "discovery_type": "consul_kv"
+ }
+}'
+```
+
+The format response as below:
+
+```json
+{
+ "node": {
+ "value": {
+ "priority": 0,
+ "update_time": 1612755230,
+ "upstream": {
+ "discovery_type": "consul_kv",
+ "service_name": "http://127.0.0.1:8500/v1/kv/upstreams/webpages/",
+ "hash_on": "vars",
+ "type": "roundrobin",
+ "pass_host": "pass"
+ },
+ "id": "1",
+ "uri": "/*",
+ "create_time": 1612755230,
+ "status": 1,
+ "saas_id": ""
+ },
+ "key": "/apisix/routes/1"
+ },
+ "action": "set"
+}
+```
+
+More usage you can find in `apisix/t/discovery/consul_kv.t` file.
Review comment:
```suggestion
You could find more usage in the `apisix/t/discovery/consul_kv.t` file.
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org