You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "coco961003 (via GitHub)" <gi...@apache.org> on 2023/09/14 08:56:50 UTC
[GitHub] [apisix] coco961003 opened a new issue, #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
coco961003 opened a new issue, #10202:
URL: https://github.com/apache/apisix/issues/10202
### Description
when I follow the tutorial for authn plug-in with Secret Manager Vault, I meet some problem.
The architecture:
apisix and other services in docker containers in one vm
vault in the same vm
```shell
[root@bogon apisix_conf]# ps -elf | grep vault
0 R root 11450 7405 0 80 0 - 28203 - 16:27 pts/1 00:00:00 grep --color=auto vault
0 S root 26173 26153 2 80 0 - 261232 futex_ 10:57 pts/2 00:09:19 vault server -dev -dev-root -token-id=root
[root@bogon apisix_conf]# netstat -apn | grep 26173
tcp 0 0 127.0.0.1:8200 0.0.0.0:* LISTEN 26173/vault
tcp 0 0 127.0.0.1:8201 0.0.0.0:* LISTEN 26173/vault
unix 2 [ ] DGRAM 154908 26173/vault
```
```shell
vault token lookup hvs.CAESIF3bN6GWO7g0ysIVmbZmo1cLU7dNMJ5QAHsW-SmW9plQGh4KHGh2cy4wRUppOUlLWHdPWTNydnZtdHA0VnZmNlc
Key Value
--- -----
accessor 2WZtBGLlCUsOhWquytkJtHGd
creation_time 1694661260
creation_ttl 768h
display_name token
entity_id n/a
expire_time 2023-10-16T11:14:20.93993443+08:00
explicit_max_ttl 0s
id hvs.CAESIF3bN6GWO7g0ysIVmbZmo1cLU7dNMJ5QAHsW-SmW9plQGh4KHGh2cy4wRUppOUlLWHdPWTNydnZtdHA0VnZmNlc
issue_time 2023-09-14T11:14:20.939938534+08:00
meta <nil>
num_uses 0
orphan false
path auth/token/create
policies [apisix-policy default]
renewable true
ttl 762h44m5s
type service
```
and some models
secret:
{
"key": "/apisix/secrets/vault/1",
"value": {
"prefix": "kv/apisix/consumer",
"create_time": 1694671488,
"id": "vault/1",
"token": "hvs.CAESIF3bN6GWO7g0ysIVmbZmo1cLU7dNMJ5QAHsW-SmW9plQGh4KHGh2cy4wRUppOUlLWHdPWTNydnZtdHA0VnZmNlc",
"update_time": 1694671488,
"uri": "https://127.0.0.1:8200"
}
}
consumer:
{
"key": "/apisix/consumers/jack",
"value": {
"username": "jack",
"plugins": {
"key-auth": {
"key": "$secret://vault/1/jack/auth-key"
}
},
"create_time": 1694671762,
"update_time": 1694671762
}
}
route:
{
"key": "/apisix/routes/1",
"value": {
"priority": 0,
"create_time": 1694672102,
"host": "testtesttest.com",
"id": "1",
"update_time": 1694672102,
"status": 1,
"plugins": {
"key-auth": {
"query": "apikey",
"header": "apikey",
"hide_credentials": false
}
},
"upstream_id": "1",
"uri": "/*"
}
}
test request(generated from postman):
curl --location 'http://192.168.137.23:9080/centerAndContent/listByUser?pageNum=1&pageSize=9&isRead=0' \
--header 'Host: testtesttest.com' \
--header 'apikey: value'
response: {"message":"Invalid API key in request"}
BTW, I am comfused when I read
"Apache APISIX 通过 Vault HTTP APIs 与 Vault 实例进行通信。必要的配置必须被添加到config.yaml中。"
and
"如果使用 APISIX Standalone 版本，则可以在 apisix.yaml 文件中添加如下配置"
and
I can config a secrect, which contains some simular info.
In fact, I removed those configuration about vault and secrets in config.yaml but create the secret above.
Thank you.
### Environment
- APISIX version and relevant services: docker-compose.yml for apache/apisix:3.5.0-debian
- device: a vm in Oracle vm virtual box
- Operating system (run `uname -a`):
Linux bogon 3.10.0-1160.80.1.el7.x86_64 #1 SMP Tue Nov 8 15:48:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] coco961003 commented on issue #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
Posted by "coco961003 (via GitHub)" <gi...@apache.org>.
coco961003 commented on issue #10202:
URL: https://github.com/apache/apisix/issues/10202#issuecomment-1719368010
> @coco961003 This looks like Apisix cannot reach Vault. You said the vault is on the VM but apisix is inside a container in the VM. Are you using host networking mode? Are you sure the apisix container can reach port 8200 on the VM?
Well, I think it is in networking mode because I can use cURL to access a website, but it cannot reach port 8200.
```shell
apisix@5bb46b874b48:/usr/local/apisix/deps/share/lua/5.1/resty$ curl --header "X-Vault-Token: $APPS_TOKEN" --request GET http://127.0.0.1:8200/v1/kv/apisix/consumer/jack
curl: (7) Failed to connect to 127.0.0.1 port 8200: Connection refused
apisix@5bb46b874b48:/usr/local/apisix/deps/share/lua/5.1/resty$ curl -X GET http://baidu.com
<html>
<meta http-equiv="refresh" content="0;url=http://www.baidu.com/">
</html>
```
Could you give me some suggestions?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] coco961003 commented on issue #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
Posted by "coco961003 (via GitHub)" <gi...@apache.org>.
coco961003 commented on issue #10202:
URL: https://github.com/apache/apisix/issues/10202#issuecomment-1720354207
> Can you show the docker compose you used?
I followed this tutorial.
[APISIX v3.5 Installation](https://apisix.apache.org/docs/apisix/installation-guide/)
docker-compose.yml
```dockerfile
#
# Apache 2.0 license
#
version: "3"
services:
apisix-dashboard:
image: apache/apisix-dashboard:3.0.1-alpine
restart: always
volumes:
- ./dashboard_conf/conf.yaml:/usr/local/apisix-dashboard/conf/conf.yaml
ports:
- "9000:9000"
networks:
apisix:
apisix:
image: apache/apisix:${APISIX_IMAGE_TAG:-3.5.0-debian}
restart: always
volumes:
- ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro
depends_on:
- etcd
##network_mode: host
ports:
- "9180:9180/tcp"
- "9080:9080/tcp"
- "9091:9091/tcp"
- "9443:9443/tcp"
- "9092:9092/tcp"
networks:
apisix:
etcd:
image: bitnami/etcd:3.4.15
restart: always
volumes:
- etcd_data:/bitnami/etcd
environment:
ETCD_ENABLE_V2: "true"
ALLOW_NONE_AUTHENTICATION: "yes"
ETCD_ADVERTISE_CLIENT_URLS: "http://etcd:2379"
ETCD_LISTEN_CLIENT_URLS: "http://0.0.0.0:2379"
ports:
- "2379:2379/tcp"
networks:
apisix:
web1:
image: nginx:1.19.0-alpine
restart: always
volumes:
- ./upstream/web1.conf:/etc/nginx/nginx.conf
ports:
- "9081:80/tcp"
environment:
- NGINX_PORT=80
networks:
apisix:
web2:
image: nginx:1.19.0-alpine
restart: always
volumes:
- ./upstream/web2.conf:/etc/nginx/nginx.conf
ports:
- "9082:80/tcp"
environment:
- NGINX_PORT=80
networks:
apisix:
prometheus:
image: prom/prometheus:v2.25.0
restart: always
volumes:
- ./prometheus_conf/prometheus.yml:/etc/prometheus/prometheus.yml
ports:
- "9090:9090"
networks:
apisix:
grafana:
image: grafana/grafana:7.3.7
restart: always
ports:
- "3000:3000"
volumes:
- "./grafana_conf/provisioning:/etc/grafana/provisioning"
- "./grafana_conf/dashboards:/var/lib/grafana/dashboards"
- "./grafana_conf/config/grafana.ini:/etc/grafana/grafana.ini"
networks:
apisix:
networks:
apisix:
driver: bridge
volumes:
etcd_data:
driver: local
```
config.yaml for APISIX
```dockerfile
#
# Apache 2.0 license
#
apisix:
node_listen: 9080 # APISIX listening port
enable_ipv6: false
enable_control: true
control:
ip: "0.0.0.0"
port: 9092
deployment:
admin:
allow_admin: # https://nginx.org/en/docs/http/ngx_http_access_module.html#allow
- 0.0.0.0/0 # We need to restrict ip access rules for security. 0.0.0.0/0 is for test.
admin_key:
- name: "admin"
key: edd1c9f034335f136f87ad84b625c8f1
role: admin # admin: manage all configuration data
- name: "viewer"
key: 4054f7cf07e344346cd3f287985e76a2
role: viewer
etcd:
host: # it's possible to define multiple etcd hosts addresses of the same etcd cluster.
- "http://etcd:2379" # multiple etcd address
prefix: "/apisix" # apisix configurations prefix
timeout: 30 # 30 seconds
plugin_attr:
prometheus:
export_addr:
ip: "0.0.0.0"
port: 9091
nginx_config:
error_log_level: "info"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] coco961003 closed issue #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
Posted by "coco961003 (via GitHub)" <gi...@apache.org>.
coco961003 closed issue #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
URL: https://github.com/apache/apisix/issues/10202
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Revolyssup commented on issue #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on issue #10202:
URL: https://github.com/apache/apisix/issues/10202#issuecomment-1720688378
@coco961003 Apisix is inside the apisix docker network and vault is outside so they can't connect. You can either add vault inside docker-compose and attach it to the apisix network like other services in docker compose or you can run apisix service in host networking mode(look at the commented out line in your docker compose `network_mode: host`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault [apisix]
Posted by "dahmoulayebrahim (via GitHub)" <gi...@apache.org>.
dahmoulayebrahim commented on issue #10202:
URL: https://github.com/apache/apisix/issues/10202#issuecomment-2009403011
Hi, I have the same problem with vault, I always get {"message":"Invalid API key in request"}.
can you please show me how you've solved it?
docker compse file:
apisix:
image: apache/apisix:${APISIX_IMAGE_TAG:-3.8.0-debian}
restart: always
volumes:
- ./apisix_conf/config.yaml:/usr/local/apisix/conf/config.yaml:ro
depends_on:
- etcd
# network_mode: host
ports:
- "9180:9180/tcp"
- "9080:9080/tcp"
- "9091:9091/tcp"
- "9443:9443/tcp"
- "9092:9092/tcp"
networks:
- apisix
vault:
image: vault:1.13.3
environment:
VAULT_ADDR: "http://127.0.0.1:8200"
ports:
- "8200:8200"
volumes:
- ./vault/file:/vault/file:rw
- ./vault/config:/vault/config:rw
cap_add:
- IPC_LOCK
entrypoint: vault server -config=/vault/config/vault.json
networks:
- apisix
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] coco961003 commented on issue #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
Posted by "coco961003 (via GitHub)" <gi...@apache.org>.
coco961003 commented on issue #10202:
URL: https://github.com/apache/apisix/issues/10202#issuecomment-1720925577
@Revolyssup Well, It seems that it doesn't work if make host networking mode enable directly.
```shell
[root@bogon example]# docker compose -p docker-apisix up -d
[+] Running 7/9
⠿ Network docker-apisix_apisix Created 0.1s
⠿ Container docker-apisix-apisix-dashboard-1 Started 3.8s
⠿ Container docker-apisix-etcd-1 Started 3.5s
⠿ Container docker-apisix-web1-1 Started 3.4s
⠿ Container docker-apisix-web2-1 Started 3.7s
⠿ Container docker-apisix-prometheus-1 Started 3.7s
⠿ Container docker-apisix-grafana-1 Started 3.4s
⠿ Container docker-apisix-apisix-1 Starting 4.0s
⠏ apisix Published ports are discarded when using host network mode 0.0s
Error response from daemon: failed to add interface vethdfc4c8a to sandbox: error setting interface "vethdfc4c8a" IP to 172.26.0.8/16: cannot program address 172.26.0.8/16 in sandbox interface because it conflicts with existing route {Ifindex: 235 Dst: 172.26.0.0/16 Src: 172.26.0.1 Gw: <nil> Flags: [] Table: 254}
```
Where can I get more information?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Revolyssup commented on issue #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on issue #10202:
URL: https://github.com/apache/apisix/issues/10202#issuecomment-1719302222
@coco961003 This looks like Apisix cannot reach Vault. You said the vault is on the VM but apisix is inside a container in the VM. Are you using host networking mode? Are you sure the apisix container can reach port 8200 on the VM?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Revolyssup commented on issue #10202: help request: how to solve the problem CONNECTION REFUSED, between APISIX and Vault
Posted by "Revolyssup (via GitHub)" <gi...@apache.org>.
Revolyssup commented on issue #10202:
URL: https://github.com/apache/apisix/issues/10202#issuecomment-1719377580
Can you show the docker compose you used?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org