You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/09/06 03:49:08 UTC
[GitHub] [apisix] Nobilta opened a new issue, #7867: help request: can i modify the config_etcd.lua's nginx timer?
Nobilta opened a new issue, #7867:
URL: https://github.com/apache/apisix/issues/7867
### Description
my apisix's etcd cluster have to many watch request(6000 connections per second),so i want to reduce the pressure of etcd cluster.if i can modify the **config_etcd.lua**'s timer of new function timer from 0 to 0.5 or 1 like this?(i have a test,and it can reduce the number of watch useful ,but i don't know if this can due to a unexpected error)or if have better method to reduce watch?
```
function _M.new(key, opts)
...
if automatic then
if not key then
return nil, "missing `key` argument"
end
if loaded_configuration[key] then
local res = loaded_configuration[key]
loaded_configuration[key] = nil -- tried to load
log.notice("use loaded configuration ", key)
local dir_res, headers = res.body, res.headers
load_full_data(obj, dir_res, headers)
end
ngx_timer_at(1, _automatic_fetch, obj)
else
local etcd_cli, err = get_etcd()
if not etcd_cli then
return nil, "failed to start a etcd instance: " .. err
end
obj.etcd_cli = etcd_cli
end
if key then
created_obj[key] = obj
end
return obj
end
```
### Environment
- APISIX version (run `apisix version`):2.15
- Operating system (run `uname -a`):
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`):3.5
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
- LuaRocks version, for installation issues (run `luarocks --version`):
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Nobilta commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
Nobilta commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1237962673
> I don't think your change will work, you just delay the action of fetch data, not the period of fetch data, it don't reduce the number of requests to etcd. Could you tell me why you have so many requests to etcd?
@monkeyDluffy6017 we have many apisix gateway(up to 20),and we just have 3 etcd server(as a cluster),so the pressure of etcd is too high.so do you have a better solution of this condition?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Nobilta closed issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
Nobilta closed issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
URL: https://github.com/apache/apisix/issues/7867
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Nobilta commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
Nobilta commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238820002
> ```
> modify the config_etcd.lua's timer of new function timer from 0 to 0.5 or 1
> ```
>
> How did you test ？Does this really help? @Nobilta
in our test ,although the total number of request to etcd is not reduce,the number indeed reduced in the unit time(same request but they use more time,so in the unit time,request has be reduced).so if we can came to a conclusion that it was useful and will not due to a bad result?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238958086
@Nobilta Can we talk with live chat software? we are interested in your etcd problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Nobilta commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
Nobilta commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238006714
>
in future,we will expand our apisix cluster triple,so the etcd's pressure will be more,and i worry the Stablize of etcd cluster
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238222634
```
modify the config_etcd.lua's timer of new function timer from 0 to 0.5 or 1
```
How did you test ？Does this really help? @Nobilta
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Nobilta commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
Nobilta commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238840586
> Could you try to use a random time instead of 0.5 or 1 ? all requests was sent at the same time may have more pressure on etcd
good suggestion.we will do more test,thank you very much!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1237953706
I don't think your change will work, you just delay the action of fetch data, not the period of fetch data, it don't reduce the number of requests to etcd.
Could you tell me why you have so many requests to etcd?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Nobilta commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
Nobilta commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238006919
> How many workers does a apisix have ? How many resources are you watching ?
in future,we will expand our apisix cluster triple,so the etcd's pressure will be more,and i worry the Stablize of etcd cluster
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] Nobilta commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
Nobilta commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238817977
> > > How many workers does a apisix have ? How many resources are you watching ?
> >
> >
> > in future,we will expand our apisix cluster triple,so the etcd's pressure will be more,and i worry the Stablize of etcd cluster
>
> A better way is adding another intermediate component to convert the http connections to grpc connections.And it's stateless so that you can scale it. You may need to use gRPC gateway feature for it .
do you have any recommendations about open source gRPC gateway or we should build a gateway by ourself?thank you very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238032529
> > How many workers does a apisix have ? How many resources are you watching ?
>
>
>
> in future,we will expand our apisix cluster triple,so the etcd's pressure will be more,and i worry the Stablize of etcd cluster
A better way is adding another intermediate component to convert the http connections to grpc connections.And it's stateless so that you can scale it. You may need to use gRPC gateway feature for it .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1237971932
How many workers does a apisix have ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on issue #7867: help request: can i modify the config_etcd.lua's nginx timer?
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on issue #7867:
URL: https://github.com/apache/apisix/issues/7867#issuecomment-1238824172
Couly you try to use a random time instead of 0.5 or 1 ? all requests was sent at the same time may have more pressure on etcd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org