You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by "kingluo (via GitHub)" <gi...@apache.org> on 2023/05/06 08:19:36 UTC
[GitHub] [apisix] kingluo opened a new pull request, #9420: fix(etcd): reuse cli and enable keepalive
kingluo opened a new pull request, #9420:
URL: https://github.com/apache/apisix/pull/9420
### Description
<!-- Please include a summary of the change and which issue is fixed. -->
<!-- Please also include relevant motivation and context. -->
1. The JWT auth token is stored in the etcd cli table, so reuse the table between calls to avoid repeat auth calls.
2. The path between conf server and etcd should enable keepalive so that we could reuse the same connection to do etcd calls. Besides avoiding connection overhead, the etcd v3 uses [connection based auth](https://etcd.io/docs/v3.5/learning/design-auth-v3/).
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [ ] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [x] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingluo commented on a diff in pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "kingluo (via GitHub)" <gi...@apache.org>.
kingluo commented on code in PR #9420:
URL: https://github.com/apache/apisix/pull/9420#discussion_r1196119103
##########
t/core/config_etcd.t:
##########
@@ -266,7 +266,6 @@ qr/etcd auth failed/
etcd auth failed
etcd auth failed
etcd auth failed
-etcd auth failed
Review Comment:
yes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingluo commented on a diff in pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "kingluo (via GitHub)" <gi...@apache.org>.
kingluo commented on code in PR #9420:
URL: https://github.com/apache/apisix/pull/9420#discussion_r1195000776
##########
t/core/config_etcd.t:
##########
@@ -233,44 +233,7 @@ passed
-=== TEST 7: ensure only one auth request per subsystem for all the etcd sync
---- yaml_config
-apisix:
- node_listen: 1984
-deployment:
- role: traditional
- role_traditional:
- config_provider: etcd
- etcd:
- host:
- - "http://127.0.0.1:1980" -- fake server port
- timeout: 1
- user: root # root username for etcd
- password: 5tHkHhYkjr6cQY # root password for etcd
---- extra_init_by_lua
-local health_check = require("resty.etcd.health_check")
-health_check.get_target_status = function()
- return true
-end
---- config
- location /t {
- content_by_lua_block {
- ngx.sleep(0.5)
- }
- }
---- request
-GET /t
---- grep_error_log eval
-qr/etcd auth failed/
---- grep_error_log_out
-etcd auth failed
Review Comment:
No need to prove reused or not in this test case.
If we need a meaningless test case, we could hook the get_etcd_cli to check if the cli is reused, but we could prove it in reality based on the fact that the performance is improved, which is hard to express via test::nginx. So we just do not write a new test case for this PR.
Back to this test case, it is arbitrary and does not describe what is "subsystem".
It's useless and confusing. If you could explain it, then I will keep it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #9420:
URL: https://github.com/apache/apisix/pull/9420#discussion_r1194967628
##########
t/core/config_etcd.t:
##########
@@ -233,44 +233,7 @@ passed
-=== TEST 7: ensure only one auth request per subsystem for all the etcd sync
---- yaml_config
-apisix:
- node_listen: 1984
-deployment:
- role: traditional
- role_traditional:
- config_provider: etcd
- etcd:
- host:
- - "http://127.0.0.1:1980" -- fake server port
- timeout: 1
- user: root # root username for etcd
- password: 5tHkHhYkjr6cQY # root password for etcd
---- extra_init_by_lua
-local health_check = require("resty.etcd.health_check")
-health_check.get_target_status = function()
- return true
-end
---- config
- location /t {
- content_by_lua_block {
- ngx.sleep(0.5)
- }
- }
---- request
-GET /t
---- grep_error_log eval
-qr/etcd auth failed/
---- grep_error_log_out
-etcd auth failed
Review Comment:
Why remove this? how do you prove that the cli is reused? one log can prove that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] leslie-tsang commented on a diff in pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "leslie-tsang (via GitHub)" <gi...@apache.org>.
leslie-tsang commented on code in PR #9420:
URL: https://github.com/apache/apisix/pull/9420#discussion_r1195822571
##########
t/core/config_etcd.t:
##########
@@ -233,44 +233,7 @@ passed
-=== TEST 7: ensure only one auth request per subsystem for all the etcd sync
---- yaml_config
-apisix:
- node_listen: 1984
-deployment:
- role: traditional
- role_traditional:
- config_provider: etcd
- etcd:
- host:
- - "http://127.0.0.1:1980" -- fake server port
- timeout: 1
- user: root # root username for etcd
- password: 5tHkHhYkjr6cQY # root password for etcd
---- extra_init_by_lua
-local health_check = require("resty.etcd.health_check")
-health_check.get_target_status = function()
- return true
-end
---- config
- location /t {
- content_by_lua_block {
- ngx.sleep(0.5)
- }
- }
---- request
-GET /t
---- grep_error_log eval
-qr/etcd auth failed/
---- grep_error_log_out
-etcd auth failed
Review Comment:
We need a test case to make sure the fix work as expected. :)
Plz add a new test case to prove it.
##########
t/core/config_etcd.t:
##########
@@ -233,44 +233,7 @@ passed
-=== TEST 7: ensure only one auth request per subsystem for all the etcd sync
---- yaml_config
-apisix:
- node_listen: 1984
-deployment:
- role: traditional
- role_traditional:
- config_provider: etcd
- etcd:
- host:
- - "http://127.0.0.1:1980" -- fake server port
- timeout: 1
- user: root # root username for etcd
- password: 5tHkHhYkjr6cQY # root password for etcd
---- extra_init_by_lua
-local health_check = require("resty.etcd.health_check")
-health_check.get_target_status = function()
- return true
-end
---- config
- location /t {
- content_by_lua_block {
- ngx.sleep(0.5)
- }
- }
---- request
-GET /t
---- grep_error_log eval
-qr/etcd auth failed/
---- grep_error_log_out
-etcd auth failed
Review Comment:
BTW, it‘s weird to delete a test case in a fix PR without adding any new test case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #9420:
URL: https://github.com/apache/apisix/pull/9420#discussion_r1194967628
##########
t/core/config_etcd.t:
##########
@@ -233,44 +233,7 @@ passed
-=== TEST 7: ensure only one auth request per subsystem for all the etcd sync
---- yaml_config
-apisix:
- node_listen: 1984
-deployment:
- role: traditional
- role_traditional:
- config_provider: etcd
- etcd:
- host:
- - "http://127.0.0.1:1980" -- fake server port
- timeout: 1
- user: root # root username for etcd
- password: 5tHkHhYkjr6cQY # root password for etcd
---- extra_init_by_lua
-local health_check = require("resty.etcd.health_check")
-health_check.get_target_status = function()
- return true
-end
---- config
- location /t {
- content_by_lua_block {
- ngx.sleep(0.5)
- }
- }
---- request
-GET /t
---- grep_error_log eval
-qr/etcd auth failed/
---- grep_error_log_out
-etcd auth failed
Review Comment:
Why remove this? how do you prove that the cli is reused?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingluo commented on a diff in pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "kingluo (via GitHub)" <gi...@apache.org>.
kingluo commented on code in PR #9420:
URL: https://github.com/apache/apisix/pull/9420#discussion_r1191428490
##########
t/core/config_etcd.t:
##########
@@ -233,44 +233,7 @@ passed
-=== TEST 7: ensure only one auth request per subsystem for all the etcd sync
---- yaml_config
-apisix:
- node_listen: 1984
-deployment:
- role: traditional
- role_traditional:
- config_provider: etcd
- etcd:
- host:
- - "http://127.0.0.1:1980" -- fake server port
- timeout: 1
- user: root # root username for etcd
- password: 5tHkHhYkjr6cQY # root password for etcd
---- extra_init_by_lua
-local health_check = require("resty.etcd.health_check")
-health_check.get_target_status = function()
- return true
-end
---- config
- location /t {
- content_by_lua_block {
- ngx.sleep(0.5)
- }
- }
---- request
-GET /t
---- grep_error_log eval
-qr/etcd auth failed/
---- grep_error_log_out
-etcd auth failed
Review Comment:
no duplicated auth anymore, due to cli reused.
##########
t/core/etcd.t:
##########
@@ -401,35 +401,3 @@ qr/init_by_lua:\d+: \S+/
init_by_lua:12: ab
init_by_lua:19: 200
init_by_lua:26: 404
-
-
-
-=== TEST 8: error handling in server_version
---- yaml_config
-deployment:
- role: traditional
- role_traditional:
- config_provider: etcd
- etcd:
- host:
- - "http://127.0.0.1:2379"
- prefix: "/apisix"
---- config
- location /t {
- content_by_lua_block {
- local etcd_lib = require("resty.etcd")
- -- the mock won't take effect when using gRPC because the connection will be cached
- etcd_lib.new = function()
Review Comment:
hijacking `resty.etcd.new` does not work after the cli is created once, because it's reused afterwards.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #9420:
URL: https://github.com/apache/apisix/pull/9420#discussion_r1196067816
##########
t/core/config_etcd.t:
##########
@@ -266,7 +266,6 @@ qr/etcd auth failed/
etcd auth failed
etcd auth failed
etcd auth failed
-etcd auth failed
Review Comment:
Does this test case prove that the cli is reused?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 merged pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "monkeyDluffy6017 (via GitHub)" <gi...@apache.org>.
monkeyDluffy6017 merged PR #9420:
URL: https://github.com/apache/apisix/pull/9420
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kingluo commented on a diff in pull request #9420: fix(etcd): reuse cli and enable keepalive
Posted by "kingluo (via GitHub)" <gi...@apache.org>.
kingluo commented on code in PR #9420:
URL: https://github.com/apache/apisix/pull/9420#discussion_r1195894843
##########
t/core/config_etcd.t:
##########
@@ -233,44 +233,7 @@ passed
-=== TEST 7: ensure only one auth request per subsystem for all the etcd sync
---- yaml_config
-apisix:
- node_listen: 1984
-deployment:
- role: traditional
- role_traditional:
- config_provider: etcd
- etcd:
- host:
- - "http://127.0.0.1:1980" -- fake server port
- timeout: 1
- user: root # root username for etcd
- password: 5tHkHhYkjr6cQY # root password for etcd
---- extra_init_by_lua
-local health_check = require("resty.etcd.health_check")
-health_check.get_target_status = function()
- return true
-end
---- config
- location /t {
- content_by_lua_block {
- ngx.sleep(0.5)
- }
- }
---- request
-GET /t
---- grep_error_log eval
-qr/etcd auth failed/
---- grep_error_log_out
-etcd auth failed
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org