You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "wjchan89 (via GitHub)" <gi...@apache.org> on 2025/06/17 09:34:06 UTC
[I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
wjchan89 opened a new issue, #12341:
URL: https://github.com/apache/apisix/issues/12341
### Current Behavior
**Single-Cluster Mode is OK**
discovery:
kubernetes:
- id: aaa
client:
token: xxx
service:
schema: https
port: '6443'
host: 192.168.30.99
curl http://127.0.0.1:9080/apisix/admin/routes/testaaa \
-H 'X-API-KEY: xxx' -X PUT -d '
{
"methods": ["GET"],
"uri": "/testaaa",
"upstream": {
"discovery_type": "kubernetes",
"service_name": "aaa/namespace/appa:tcp26160"
}
}'
curl http://localhost:9081/testaaa
{"timestamp":"2025-06-17T09:06:13.478+00:00","status":404,"error":"Not Found","message":"","path":"/testaaa"}
**Multi-Cluster Mode is not working**
discovery:
kubernetes:
- id: aaa
client:
token: xxx
service:
schema: https
port: '6443'
host: 192.168.30.99
- id: bbb
client:
token: xxx
service:
schema: https
port: '6443'
host: 192.168.30.89
curl http://127.0.0.1:9080/apisix/admin/routes/testaaa \
-H 'X-API-KEY: xxx' -X PUT -d '
{
"methods": ["GET"],
"uri": "/testaaa",
"upstream": {
"discovery_type": "kubernetes",
"service_name": "aaa/namespace/appa:tcp26160"
}
}'
curl http://localhost:9081/testaaa -i
HTTP/1.1 503 Service Temporarily Unavailable
Date: Tue, 17 Jun 2025 09:10:30 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 225
Connection: keep-alive
Server: APISIX/3.2.2
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>openresty</center>
<p><em>Powered by APISIX.</em></p></body>
</html>
error log:
2025/06/17 17:10:30 [error] 103307#103307: *2653 [lua] init.lua:493: handle_upstream(): failed to set upstream: no valid upstream node: nil, client: ::1, server: _, request: "GET /testaaa HTTP/1.1", host: "localhost:9081"
### Expected Behavior
how to config in Multi-Cluster Mode?
### Error Logs
2025/06/17 17:10:30 [error] 103307#103307: *2653 [lua] init.lua:493: handle_upstream(): failed to set upstream: no valid upstream node: nil, client: ::1, server: _, request: "GET /testaaa HTTP/1.1", host: "localhost:9081"
### Steps to Reproduce
1. config config.yaml
2. restart apisix
3. config route and sent request
### Environment
- APISIX version (run `apisix version`):3.2
- Operating system (run `uname -a`):
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`):
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop commented on issue #12341:
URL: https://github.com/apache/apisix/issues/12341#issuecomment-2983562638
> test in apisix 3.2 and 3.12
Can you try using the control API dump data in 3.12?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop commented on issue #12341:
URL: https://github.com/apache/apisix/issues/12341#issuecomment-2995042898
Hi @wjchan89, any updates?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop closed issue #12341: bug: multi-cluster mode Kubernetes service discovery is not working
URL: https://github.com/apache/apisix/issues/12341
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop commented on issue #12341:
URL: https://github.com/apache/apisix/issues/12341#issuecomment-2982485135
Are there any other error logs?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
Posted by "wjchan89 (via GitHub)" <gi...@apache.org>.
wjchan89 commented on issue #12341:
URL: https://github.com/apache/apisix/issues/12341#issuecomment-2983525626
> Are there any other error logs?
There are no more logs.
But,if I delete the appa from one cluster,it work.
Different Cluster ID in apisix , same namespace and same service in two cluster, not working. test in apisix 3.2 and 3.12
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop commented on issue #12341:
URL: https://github.com/apache/apisix/issues/12341#issuecomment-3116930289
If there is still a problem, please open it again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
Posted by "wjchan89 (via GitHub)" <gi...@apache.org>.
wjchan89 commented on issue #12341:
URL: https://github.com/apache/apisix/issues/12341#issuecomment-2982455879
> Hi [@wjchan89](https://github.com/wjchan89), are there any other error logs in apisix? We need to make sure that apisix can access the cluster normally. You can also check the service discovery data in apisix with the control api https://apisix.apache.org/docs/apisix/control-api/#get-v1discoveryservicedump
apisix version 3.2,there is no such control api for discovery.
when I set one cluster like this,
```bash
discovery:
kubernetes:
- id: aaa
client:
token: xxx
service:
schema: https
port: '6443'
host: 192.168.30.99
```
the routes:
```bash
curl http: //127.0.0.1:9090/v1/routes
[
{
"has_domain": false,
"orig_modifiedIndex": 185,
"createdIndex": 68,
"modifiedIndex": 185,
"value": {
"create_time": 1750146490,
"methods": [
"GET"
],
"uri": "/flowRuleA",
"update_time": 1750214026,
"id": "testaaa",
"upstream": {
"original_nodes": [
{
"port": 26160,
"weight": 50,
"host": "192.168.30.124",
"metadata": {
"pod-template-hash": "6f5c7b5c47",
"version": "v1",
"app": "appa"
}
}
],
"hash_on": "vars",
"service_name": "aaa/namespace/appa:tcp26160",
"type": "roundrobin",
"pass_host": "pass",
"nodes": [
{
"metadata": {
"pod-template-hash": "6f5c7b5c47",
"version": "v1",
"app": "appa"
},
"port": 26160,
"upstream_host": "192.168.30.124:26160",
"priority": 0,
"host": "192.168.30.124",
"weight": 50
}
],
"discovery_type": "kubernetes",
"scheme": "http",
"nodes_ref": [
{
"upstream_host": "192.168.30.124:26160",
"priority": 0,
"port": 26160,
"weight": 50,
"host": "192.168.30.124",
"metadata": {
"pod-template-hash": "6f5c7b5c47",
"version": "v1",
"app": "appa"
}
}
]
},
"priority": 0,
"status": 1
},
"clean_handlers": {},
"key": "/apisix/routes/testaaa"
},
{
"has_domain": false,
"orig_modifiedIndex": 142,
"createdIndex": 119,
"modifiedIndex": 142,
"value": {
"update_time": 1750150991,
"create_time": 1750148686,
"priority": 0,
"id": "testbbb",
"uri": "/testbbb",
"upstream": {
"pass_host": "pass",
"service_name": "bbb/namespace/appa:tcp8080",
"hash_on": "vars",
"discovery_type": "kubernetes",
"scheme": "http",
"type": "roundrobin"
},
"methods": [
"GET"
],
"status": 1
},
"clean_handlers": {},
"key": "/apisix/routes/testbbb"
}
]
```
when I set two clusters like this,
```bash
discovery:
kubernetes:
- id: aaa
client:
token: xxx
service:
schema: https
port: '6443'
host: 192.168.30.99
- id: bbb
client:
token: xxx
service:
schema: https
port: '6443'
host: 192.168.30.89
```
the routes:
```bash
curl http: //127.0.0.1:9090/v1/routes
[
{
"createdIndex": 68,
"clean_handlers": {},
"modifiedIndex": 120,
"key": "/apisix/routes/testaaa",
"has_domain": false,
"value": {
"methods": [
"GET"
],
"update_time": 1750148965,
"create_time": 1750146490,
"priority": 0,
"uri": "/flowRuleA",
"status": 1,
"id": "testaaa",
"upstream": {
"pass_host": "pass",
"service_name": "aaa/namespace/appa:tcp26160",
"scheme": "http",
"type": "roundrobin",
"hash_on": "vars",
"discovery_type": "kubernetes"
}
},
"orig_modifiedIndex": 120
},
{
"createdIndex": 119,
"clean_handlers": {},
"modifiedIndex": 142,
"key": "/apisix/routes/testbbb",
"has_domain": false,
"value": {
"upstream": {
"service_name": "bbb/namespace/appa:tcp8080",
"scheme": "http",
"pass_host": "pass",
"type": "roundrobin",
"hash_on": "vars",
"discovery_type": "kubernetes"
},
"update_time": 1750150991,
"create_time": 1750148686,
"methods": [
"GET"
],
"uri": "/testbbb",
"priority": 0,
"id": "testbbb",
"status": 1
},
"orig_modifiedIndex": 142
}
]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] bug: multi-cluster mode Kubernetes service discovery is not working [apisix]
Posted by "Baoyuantop (via GitHub)" <gi...@apache.org>.
Baoyuantop commented on issue #12341:
URL: https://github.com/apache/apisix/issues/12341#issuecomment-2982329623
Hi @wjchan89, are there any other error logs in apisix? We need to make sure that apisix can access the cluster normally. You can also check the service discovery data in apisix with the control api https://apisix.apache.org/docs/apisix/control-api/#get-v1discoveryservicedump
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org