You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2021/04/28 03:17:13 UTC
[GitHub] [apisix] varcher-me opened a new issue #4143: bug: prometheus plugin ask for root premission of etcd
varcher-me opened a new issue #4143:
URL: https://github.com/apache/apisix/issues/4143
### Issue description
### Environment
* apisix version (cmd: `apisix version`):2.4
* OS (cmd: `uname -a`):Linux 3.10.0-1160.15.2.el7.x86_64 (Redhat 7.9)
* OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`):openresty/1.19.3.1
* etcd version, if have (cmd: run `curl http://127.0.0.1:9090/v1/server_info` to get the info from server-info API):3.4.15
* apisix-dashboard version, if have:2.5
* luarocks version, if the issue is about installation (cmd: `luarocks --version`):
### Minimal test code / Steps to reproduce the issue
1.use etcdctl add user apisix and grant all privilege on /apisix/*
2.edit apisix/conf/config.yaml, add the user to apisix
3.run curl -i https://domainname:9443/apisix/prometheus/metrics
### What's the actual result? (including assertion message & call stack if applicable)
`HTTP/1.1 500 Internal Server Error
Date: Wed, 28 Apr 2021 03:07:52 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 174
Connection: close
Server: APISIX
<html>
<head><title>500 Internal Server Error</title></head>
<body>
<center><h1>500 Internal Server Error</h1></center>
<hr><center>openresty</center>
</body>
</html>`
### What's the expected result?
when I change the etcd user in apisix/conf/config.yaml to root, this plugin works well, like this:
`HTTP/1.1 200 OK
Date: Wed, 28 Apr 2021 03:05:20 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server: APISIX
# HELP apisix_etcd_modify_indexes Etcd modify index for APISIX keys
# TYPE apisix_etcd_modify_indexes gauge
apisix_etcd_modify_indexes{key="consumers"} 61099
apisix_etcd_modify_indexes{key="global_rules"} 60709
apisix_etcd_modify_indexes{key="max_modify_index"} 62639
apisix_etcd_modify_indexes{key="prev_index"} 62789
apisix_etcd_modify_indexes{key="protos"} 0
apisix_etcd_modify_indexes{key="routes"} 62639
apisix_etcd_modify_indexes{key="services"} 0
apisix_etcd_modify_indexes{key="ssls"} 2281
apisix_etcd_modify_indexes{key="stream_routes"} 0
apisix_etcd_modify_indexes{key="upstreams"} 61110
apisix_etcd_modify_indexes{key="x_etcd_index"} 62792
# HELP apisix_etcd_reachable Config server etcd reachable from APISIX, 0 is unreachable
# TYPE apisix_etcd_reachable gauge
apisix_etcd_reachable 1
# HELP apisix_nginx_http_current_connections Number of HTTP connections
# TYPE apisix_nginx_http_current_connections gauge
apisix_nginx_http_current_connections{state="accepted"} 1
apisix_nginx_http_current_connections{state="active"} 1
apisix_nginx_http_current_connections{state="handled"} 1
apisix_nginx_http_current_connections{state="reading"} 0
apisix_nginx_http_current_connections{state="total"} 1
apisix_nginx_http_current_connections{state="waiting"} 0
apisix_nginx_http_current_connections{state="writing"} 1
# HELP apisix_nginx_metric_errors_total Number of nginx-lua-prometheus errors
# TYPE apisix_nginx_metric_errors_total counter
apisix_nginx_metric_errors_total 0
# HELP apisix_node_info Info of APISIX node
# TYPE apisix_node_info gauge
apisix_node_info{hostname="servername"} 1`
BUT it is not correct to give apisix the root privilege.
All function and other plugins works well with the apisix user of etcd.
I think it is a bug for the plugin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] varcher-me commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
varcher-me commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828881653
> I verified that it work.
>
> my command
>
> ```shell
> etcdctl --endpoints=http://127.0.0.1:2379 user add apisix
>
> etcdctl --endpoints=http://127.0.0.1:2379 role add apisix
>
> etcdctl --endpoints=http://127.0.0.1:2379 user grant-role apisix apisix
>
> etcdctl --endpoints=http://127.0.0.1:2379 role grant-permission apisix --prefix=true readwrite /apisix
> ```
>
> my config.yaml
>
> ```yaml
> etcd:
> host:
> - "http://127.0.0.1:2379"
> prefix: "/apisix"
> user: apisix
> password: abc123
> ```
>
> the etcdctl command and return is
>
> ```shell
> etcdctl --user="apisix" --password="abc123" --endpoints=http://127.0.0.1:2379 role get apisix
> Role apisix
> KV Read:
> [/apisix, /apisiy) (prefix /apisix)
> KV Write:
> [/apisix, /apisiy) (prefix /apisix)
> ```
>
> how does this help you?
I tried this before, but seems not work. I will try it again with 2.5 later.
Thanks for help! I grant root privilege to apisix temporary in my production env.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers closed issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
tokers closed issue #4143:
URL: https://github.com/apache/apisix/issues/4143
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828184885
I know why.
In step 1, the directory permissions for the apisix user are `/apisix/`, but the prefix set in config.yaml in step2 is `/apisix`, if I also change it to `/apisix/`, it will work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] varcher-me commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
varcher-me commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828196547
> I know why.
>
> In step 1, the directory permissions for the apisix user are `/apisix/`, but the prefix set in config.yaml in step2 is `/apisix`, if I also change it to `/apisix/`, it will work.
>
> I think we should catch this exception.
In this case(change etcd path to /apisix/), the key in etcd will changed like this(checked by etcdctl), double slash. I think it still incorrect...
/apisix//consumers/
/apisix//data_plane/server_info/04ccd55b-6017-4471-bd5b-4804860fe2d5
/apisix//global_rules/
/apisix//node_status/
/apisix//plugin_configs/
/apisix//plugin_metadata/
/apisix//plugins
/apisix//plugins/
/apisix//proto/
/apisix//routes/
/apisix//services/
/apisix//ssl/
/apisix//stream_routes/
/apisix//upstreams/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828174543
I verified
step 1: use etcdctl add user apisix and grant all privilege on /apisix/*
command like
```shell
etcdctl --endpoints=http://127.0.0.1:2379 user add apisix
etcdctl --endpoints=http://127.0.0.1:2379 role add apisix
etcdctl --endpoints=http://127.0.0.1:2379 user grant-role apisix apisix
etcdctl --endpoints=http://127.0.0.1:2379 role grant-permission apisix --prefix=true readwrite /apisix/
etcdctl --endpoints=http://127.0.0.1:2379 auth enable
```
step 2: edit apisix/conf/config.yaml, add the user to apisix
the config.yaml like
```yaml
etcd:
host:
- "http://127.0.0.1:2379"
prefix: "/apisix"
user: apisix
password: abc123
```
step 3: curl metrics
command like
```shell
http://127.0.0.1:9080/apisix/prometheus/metrics
```
step 4: error.log
I get the error.log like
```nginx
2021/04/28 14:00:16 [error] 57379#1168662: *12753 [lua] http.lua:913: request_uri(): unread data in buffer, client: ::1, server: , request: "GET /apisix/prometheus/metrics HTTP/1.1", host: "localhost:9080"
2021/04/28 14:00:16 [error] 57379#1168662: *12753 lua entry thread aborted: runtime error: /usr/local/Cellar/apisix/apisix/core/etcd.lua:92: attempt to index field 'header' (a nil value)
stack traceback:
coroutine 0:
/usr/local/Cellar/apisix/apisix/core/etcd.lua: in function 'get_format'
/usr/local/Cellar/apisix/apisix/core/config_etcd.lua:75: in function 'getkey'
...cal/Cellar/apisix/apisix/plugins/prometheus/exporter.lua:310: in function 'handler'
/usr/local/Cellar/apisix/apisix/api_router.lua:130: in function 'handler'
...al/Cellar/apisix//deps/share/lua/5.1/resty/radixtree.lua:720: in function 'dispatch'
/usr/local/Cellar/apisix/apisix/api_router.lua:165: in function 'match'
/usr/local/Cellar/apisix/apisix/init.lua:292: in function 'http_access_phase'
access_by_lua(nginx.conf:209):2: in main chunk, client: ::1, server: , request: "GET /apisix/prometheus/metrics HTTP/1.1", host: "localhost:9080"
```
by the way, when I switched to apisix 2.5, I got a similar error starting apisix, which seems to be a problem with apisix and etcd interaction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass edited a comment on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
tzssangglass edited a comment on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828184885
I know why.
In step 1, the directory permissions for the apisix user are `/apisix/`, but the prefix set in config.yaml in step2 is `/apisix`, if I also change it to `/apisix/`, it will work.
I think we should catch this exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828264983
I verified that it work.
my command
```shell
etcdctl --endpoints=http://127.0.0.1:2379 user add apisix
etcdctl --endpoints=http://127.0.0.1:2379 role add apisix
etcdctl --endpoints=http://127.0.0.1:2379 user grant-role apisix apisix
etcdctl --endpoints=http://127.0.0.1:2379 role grant-permission apisix --prefix=true readwrite /apisix
```
my config.yaml
```yaml
etcd:
host:
- "http://127.0.0.1:2379"
prefix: "/apisix"
user: apisix
password: abc123
```
the etcdctl command and return is
```shell
etcdctl --user="apisix" --password="abc123" --endpoints=http://127.0.0.1:2379 role get apisix
Role apisix
KV Read:
[/apisix, /apisiy) (prefix /apisix)
KV Write:
[/apisix, /apisiy) (prefix /apisix)
```
how does this help you?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] varcher-me commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
varcher-me commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828222795
> at step 1, when granting directory permissions, the command should be something like this
>
> ```shell
> etcdctl --endpoints=http://127.0.0.1:2379 role grant-permission apisix --prefix=true readwrite /apisix
> ```
>
> use `/apisix` not `/apisix/`
Yes, I've already tried grant /apisix prefix to the user "apisix", problem still.
the etcdctl command and return is here.
`etcdctl role get role-apisix --user=root`
(And enter password)
Role role-apisix
KV Read:
/apisix
[/apisix, /apisiy) (prefix /apisix)
[/apisix/, /apisix0) (prefix /apisix/)
/apisix/*
KV Write:
/apisix
[/apisix, /apisiy) (prefix /apisix)
[/apisix/, /apisix0) (prefix /apisix/)
/apisix/*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828205871
at step 1, when granting directory permissions, the command should be something like this
```shell
etcdctl --endpoints=http://127.0.0.1:2379 role grant-permission apisix --prefix=true readwrite /apisix
```
use `/apisix` not `/apisix/`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828175547
@spacewander @tokers pls take a look
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #4143: bug: prometheus plugin ask for root premission of etcd
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #4143:
URL: https://github.com/apache/apisix/issues/4143#issuecomment-828332133
> I know why.
>
> In step 1, the directory permissions for the apisix user are `/apisix/`, but the prefix set in config.yaml in step2 is `/apisix`, if I also change it to `/apisix/`, it will work.
>
> I think we should catch this exception.
Yes, we should capture this exception and output some more friendly hints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org