You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/03/09 01:59:16 UTC
[GitHub] [apisix] zhendongcmss opened a new issue #6555: bug: Prometheus metrics igress and egress is inaccurate
zhendongcmss opened a new issue #6555:
URL: https://github.com/apache/apisix/issues/6555
### Issue description
There two routes, one priority is 0 route id = 1, anther is 30 route id = 2.
if request hit route 1, the prometheus metric igress and egress is accurate, but if request hit route 2, the igress and egress valuse is double of bandwidth.
I think that apisix records twice `request_length` and `bytes_sent`.
```
metrics.bandwidth:inc(vars.request_length,
gen_arr("ingress", route_id, service_id, consumer_name, balancer_ip))
metrics.bandwidth:inc(vars.bytes_sent,
gen_arr("egress", route_id, service_id, consumer_name, balancer_ip))
```
### Environment
- apisix version (cmd: `apisix version`): 2.7
- OS (cmd: `uname -a`):
- OpenResty / Nginx version (cmd: `nginx -V` or `openresty -V`):
- etcd version, if have (cmd: run `curl http://127.0.0.1:9090/v1/server_info` to get the info from server-info API):
- apisix-dashboard version, if have:
- the plugin runner version, if the issue is about a plugin runner (cmd: depended on the kind of runner):
- luarocks version, if the issue is about installation (cmd: `luarocks --version`):
### Steps to reproduce
route configuration
```
{
"action": "get",
"node": {
"key": "/apisix/routes",
"dir": true,
"nodes": [
{
"key": "/apisix/routes/1",
"modifiedIndex": 6332825,
"value": {
"status": 1,
"methods": [
"PUT",
"GET",
"POST",
"DELETE",
"PATCH",
"HEAD",
"OPTIONS",
"CONNECT",
"TRACE"
],
"upstream": {
"pass_host": "pass",
"nodes": {
"10.235.79.7:8079": 10
},
"scheme": "http",
"hash_on": "vars",
"type": "roundrobin"
},
"uri": "/*",
"create_time": 1646098895,
"id": "1",
"priority": 0,
"update_time": 1646098895,
"plugins": {
"prometheus": {
"prefer_name": false
}
}
},
"createdIndex": 6332825
},
{
"key": "/apisix/routes/3",
"modifiedIndex": 4020996,
"value": {
"status": 1,
"methods": [
"PUT",
"GET",
"POST",
"DELETE",
"PATCH",
"HEAD",
"OPTIONS",
"CONNECT",
"TRACE"
],
"upstream": {
"hash_on": "vars",
"type": "roundrobin",
"nodes": {
"10.235.79.7:9096": 10
},
"pass_host": "pass",
"scheme": "http"
},
"uri": "/*",
"priority": 20,
"vars": [
"OR",
[
"host",
"~*",
".*(abc.cn)$"
],
[
"host",
"==",
"10.235.82.1"
]
],
"id": "3",
"update_time": 1643262492,
"create_time": 1638353491
},
"createdIndex": 1682
}
]
}
}
```
Client write: 145MB/s + 133MB/s + 144MB/s + 1.15GB/s = 1.55GB/s, but apisix igress is 2.85GB/s
### Actual result
The bandwidth = apisix igress/egress
### Error log
no
### Expected result
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] shuaijinchao commented on issue #6555: bug: Prometheus metrics igress and egress is inaccurate
Posted by GitBox <gi...@apache.org>.
shuaijinchao commented on issue #6555:
URL: https://github.com/apache/apisix/issues/6555#issuecomment-1064289873
@zhendongcmss when `Global Rule` and `route` enabled `prometheus` plugin at the same time, it will cause repeated reporting. we should merge the same plugin on the `Global Rule` and the `route`.
cc @spacewander
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on issue #6555: bug: Prometheus metrics igress and egress is inaccurate
Posted by GitBox <gi...@apache.org>.
spacewander commented on issue #6555:
URL: https://github.com/apache/apisix/issues/6555#issuecomment-1064699538
> @zhendongcmss when `Global Rule` and `route` enabled `prometheus` plugin at the same time, it will cause repeated reporting. we should merge the same plugin on the `Global Rule` and the `route`.
>
> cc @spacewander
>
>  
We can do it with
https://github.com/apache/apisix/blob/72bc1ef5a93f1d0e0dbdba52791b51843f2e45a0/apisix/plugins/skywalking.lua#L69
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] zhendongcmss edited a comment on issue #6555: bug: Prometheus metrics igress and egress is inaccurate
Posted by GitBox <gi...@apache.org>.
zhendongcmss edited a comment on issue #6555:
URL: https://github.com/apache/apisix/issues/6555#issuecomment-1064695865
Another test case is, config prometheus on global rules only. Request `http://127.0.0.1:9080`. It will hit route id =3(priority 30) ,but the host doesn't match `vars` then hit route id = 1(priority 0). At this time , watch the igress and egress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] shuaijinchao commented on issue #6555: bug: Prometheus metrics igress and egress is inaccurate
Posted by GitBox <gi...@apache.org>.
shuaijinchao commented on issue #6555:
URL: https://github.com/apache/apisix/issues/6555#issuecomment-1064870673
> Another test case is, config prometheus on global rules only. Request `http://127.0.0.1:9080`. It will hit route id =3(priority 30) ,but the host doesn't match `vars` then hit route id = 1(priority 0). At this time , watch the igress and egress.
the solution is the same.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander closed issue #6555: bug: Prometheus metrics igress and egress is inaccurate
Posted by GitBox <gi...@apache.org>.
spacewander closed issue #6555:
URL: https://github.com/apache/apisix/issues/6555
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] zhendongcmss commented on issue #6555: bug: Prometheus metrics igress and egress is inaccurate
Posted by GitBox <gi...@apache.org>.
zhendongcmss commented on issue #6555:
URL: https://github.com/apache/apisix/issues/6555#issuecomment-1064695865
Another test case is, config prometheus on global rules only. Request `http://127.0.0.1:9080`. It will hit route id =3 ,but the host doesn't match `vars` then hit route id = 1. At this time , watch the igress and egress.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org