You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/11/25 08:08:51 UTC
[GitHub] [apisix] tzssangglass opened a new pull request, #8403: feat: support global data encryption of secret information
tzssangglass opened a new pull request, #8403:
URL: https://github.com/apache/apisix/pull/8403
### Description
<!-- Please include a summary of the change and which issue is fixed. -->
<!-- Please also include relevant motivation and context. -->
Fixes # (issue)
### Checklist
- [ ] I have explained the need for this PR and the problem it solves
- [ ] I have explained the changes or the new features added to this PR
- [ ] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark the PR as draft until it's ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after making changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034684978
##########
t/node/data_encrypt.t:
##########
@@ -0,0 +1,581 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+# the sensitive data is encrypted in etcd, and it is normal to read it from the admin API
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+77+NmbYqNfN+oLm0aX5akg==
+
+
+
+=== TEST 2: enable basic auth plugin
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
Review Comment:
done
##########
t/node/data_encrypt.t:
##########
@@ -0,0 +1,581 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+# the sensitive data is encrypted in etcd, and it is normal to read it from the admin API
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+77+NmbYqNfN+oLm0aX5akg==
+
+
+
+=== TEST 2: enable basic auth plugin
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
+--- response_body
+passed
+
+
+
+=== TEST 3: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 4: multiple auth plugins work well
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ },
+ "key-auth": {
+ "key": "auth-one"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say("done")
+ }
+ }
+--- response_body
+done
+
+
+
+=== TEST 5: enable multiple auth plugins on route
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {},
+ "key-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
+--- response_body
+passed
+
+
+
+=== TEST 6: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+apikey: auth-one
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 7: disable data_encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: false
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+
+
+
+=== TEST 8: etcd store unencrypted password, enable data_encryption, decryption fails, use original password
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local core = require("apisix.core")
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res, err = core.etcd.set("/consumers/foo2", core.json.decode([[{
+ "username":"foo2",
+ "plugins":{
+ "basic-auth":{
+ "username":"foo2",
+ "password":"bar"
+ }
+ }
+ }]]))
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo2',
+ ngx.HTTP_GET
+ )
+ res = core.json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo2'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+ }
+ }
+--- response_body
+bar
+bar
+--- error_log
+failed to decrypt the conf of plugin [basic-auth] key [password], err: decrypt ssl key failed
+
+
+
+=== TEST 9: etcd stores both encrypted and unencrypted data
+# enable data_encryption, decryption of encrypted data succeeds
+# decryption of unencrypted data fails, make sure it works well
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local core = require("apisix.core")
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res, err = core.etcd.set("/consumers/foo2", core.json.decode([[{
+ "username":"foo2",
+ "plugins":{
+ "basic-auth":{
+ "username":"foo2",
+ "password":"bar"
+ },
+ "key-auth": {
+ "key": "vU/ZHVJw7b0XscDJ1Fhtig=="
+ }
+ }
+ }]]))
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo2',
+ ngx.HTTP_GET
+ )
+ res = core.json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+ ngx.say(res.value.plugins["key-auth"].key)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo2'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+ ngx.say(res.body.node.value.plugins["key-auth"].key)
+ }
+ }
+--- response_body
+bar
+auth-two
+bar
+vU/ZHVJw7b0XscDJ1Fhtig==
+--- error_log
+failed to decrypt the conf of plugin [basic-auth] key [password], err: decrypt ssl key failed
+
+
+
+=== TEST 10: verify, use the foo2 consumer
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+apikey: auth-two
+Authorization: Basic Zm9vMjpiYXI=
+--- response_body
+hello world
+
+
+
+=== TEST 11: keyring rotate, encrypt with edd1c9f0985e76a2
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- response_body
+passed
+
+
+
+=== TEST 12: keyring rotate, decrypt with qeddd145sfvddff3 would fail, but encrypt with edd1c9f0985e76a2 would success
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - qeddd145sfvddff3
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 13: search consumer list
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+
+ -- dletet exist consumers
+ t('/apisix/admin/consumers/foo', ngx.HTTP_DELETE)
+ t('/apisix/admin/consumers/foo2', ngx.HTTP_DELETE)
+
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "test",
+ "plugins": {
+ "basic-auth": {
+ "username": "test",
+ "password": "test"
+ }
+ }
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ local pwds = {}
+ table.insert(pwds, res.list[1].value.plugins["basic-auth"].password)
+ table.insert(pwds, res.list[2].value.plugins["basic-auth"].password)
+
+ ngx.say(json.encode(pwds))
+ }
+ }
+--- request
+GET /t
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035511224
##########
t/node/data_encrypt2.t:
##########
@@ -0,0 +1,731 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!defined $block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ my $http_config = $block->http_config // <<_EOC_;
+ server {
+ listen 10420;
+ location /clickhouse-logger/test {
+ content_by_lua_block {
+ ngx.req.read_body()
+ local data = ngx.req.get_body_data()
+ local headers = ngx.req.get_headers()
+ ngx.log(ngx.WARN, "clickhouse body: ", data)
+ for k, v in pairs(headers) do
+ ngx.log(ngx.WARN, "clickhouse headers: " .. k .. ":" .. v)
+ end
+ ngx.say("ok")
+ }
+ }
+ location /clickhouse-logger/test1 {
+ content_by_lua_block {
+ ngx.req.read_body()
+ local data = ngx.req.get_body_data()
+ local headers = ngx.req.get_headers()
+ ngx.log(ngx.WARN, "clickhouse body: ", data)
+ for k, v in pairs(headers) do
+ ngx.log(ngx.WARN, "clickhouse headers: " .. k .. ":" .. v)
+ end
+ ngx.say("ok")
+ }
+ }
+ }
+_EOC_
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034556190
##########
docs/zh/latest/plugin-develop.md:
##########
@@ -273,6 +273,37 @@ function _M.check_schema(conf, schema_type)
end
```
+指定参数需要被加密存储（需要 APISIX 版本大于 3.0.1）
+
+有些插件需要将参数加密存储，比如 `basic-auth` 插件的 `password` 参数。这个插件需要在 `schema` 中指定哪些参数需要被加密存储。
+
+```lua
+password = { type = "string", encrypted = true },
+```
+
+通过在 `schema` 中指定 `encrypted = true`，可以将参数加密存储。APISIX 将提供以下功能：
+
+- 通过 `Admin API` 来新增和更新资源时，对于 `encrypted = true` 的参数，APISIX 会自动加密存储在 etcd 中
+- 通过 `Admin API` 来获取资源时，对于 `encrypted = true` 的参数，APISIX 会自动解密返回
+- APISIX 在使用 `encrypted = true` 的参数时，会自动解密
Review Comment:
fixed
##########
apisix/plugin.lua:
##########
@@ -901,7 +961,15 @@ _M.stream_check_schema = stream_check_schema
function _M.plugin_checker(item, schema_type)
if item.plugins then
- return check_schema(item.plugins, schema_type, true)
+ local ok, err = check_schema(item.plugins, schema_type, true)
+
+ if ok then
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034655502
##########
apisix/admin/utils.lua:
##########
@@ -78,4 +80,29 @@ function _M.fix_count(body, id)
end
+function _M.decrypt_params(decrypt_func, body, schema_type)
+ -- list
+ if body.list and #body.list > 0 then
+ for _, route in ipairs(body.list) do
+ if route.value and route.value.plugins
+ and core.table.nkeys(route.value.plugins) > 0 then
Review Comment:
We can use `core.table.isempty` for this case. In fact, we don't need to check the len of plugins.
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
+ end
+
+ return enable_data_encryption
+end
+
+
+local function get_plugin_schema(name, schema_type)
Review Comment:
```suggestion
local function get_plugin_schema_for_gde(name, schema_type)
```
would be a more appropriate name
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
+ end
+
+ return enable_data_encryption
+end
+
+
+local function get_plugin_schema(name, schema_type)
+ if not enable_gde() then
+ return false
Review Comment:
```suggestion
return nil
```
as the return type isn't boolean
##########
apisix/ssl.lua:
##########
@@ -173,7 +218,11 @@ end
local function parse_pem_priv_key(sni, pkey)
core.log.debug("parsing priv key for sni: ", sni)
- local parsed, err = ngx_ssl.parse_pem_priv_key(aes_decrypt_pkey(pkey))
+ local key, err = aes_decrypt_pkey(pkey)
+ if not key then
+ core.log.error(err)
Review Comment:
Should return from here?
##########
t/node/data_encrypt.t:
##########
@@ -0,0 +1,581 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
Review Comment:
We don't need this now (no_error_log is added by default)
##########
t/node/data_encrypt.t:
##########
@@ -0,0 +1,581 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+# the sensitive data is encrypted in etcd, and it is normal to read it from the admin API
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+77+NmbYqNfN+oLm0aX5akg==
+
+
+
+=== TEST 2: enable basic auth plugin
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
+--- response_body
+passed
+
+
+
+=== TEST 3: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 4: multiple auth plugins work well
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ },
+ "key-auth": {
+ "key": "auth-one"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say("done")
+ }
+ }
+--- response_body
+done
+
+
+
+=== TEST 5: enable multiple auth plugins on route
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {},
+ "key-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
+--- response_body
+passed
+
+
+
+=== TEST 6: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+apikey: auth-one
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 7: disable data_encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: false
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+
+
+
+=== TEST 8: etcd store unencrypted password, enable data_encryption, decryption fails, use original password
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local core = require("apisix.core")
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res, err = core.etcd.set("/consumers/foo2", core.json.decode([[{
+ "username":"foo2",
+ "plugins":{
+ "basic-auth":{
+ "username":"foo2",
+ "password":"bar"
+ }
+ }
+ }]]))
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo2',
+ ngx.HTTP_GET
+ )
+ res = core.json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo2'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+ }
+ }
+--- response_body
+bar
+bar
+--- error_log
+failed to decrypt the conf of plugin [basic-auth] key [password], err: decrypt ssl key failed
+
+
+
+=== TEST 9: etcd stores both encrypted and unencrypted data
+# enable data_encryption, decryption of encrypted data succeeds
+# decryption of unencrypted data fails, make sure it works well
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local core = require("apisix.core")
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res, err = core.etcd.set("/consumers/foo2", core.json.decode([[{
+ "username":"foo2",
+ "plugins":{
+ "basic-auth":{
+ "username":"foo2",
+ "password":"bar"
+ },
+ "key-auth": {
+ "key": "vU/ZHVJw7b0XscDJ1Fhtig=="
+ }
+ }
+ }]]))
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo2',
+ ngx.HTTP_GET
+ )
+ res = core.json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+ ngx.say(res.value.plugins["key-auth"].key)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo2'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+ ngx.say(res.body.node.value.plugins["key-auth"].key)
+ }
+ }
+--- response_body
+bar
+auth-two
+bar
+vU/ZHVJw7b0XscDJ1Fhtig==
+--- error_log
+failed to decrypt the conf of plugin [basic-auth] key [password], err: decrypt ssl key failed
+
+
+
+=== TEST 10: verify, use the foo2 consumer
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+apikey: auth-two
+Authorization: Basic Zm9vMjpiYXI=
+--- response_body
+hello world
+
+
+
+=== TEST 11: keyring rotate, encrypt with edd1c9f0985e76a2
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- response_body
+passed
+
+
+
+=== TEST 12: keyring rotate, decrypt with qeddd145sfvddff3 would fail, but encrypt with edd1c9f0985e76a2 would success
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - qeddd145sfvddff3
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 13: search consumer list
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+
+ -- dletet exist consumers
+ t('/apisix/admin/consumers/foo', ngx.HTTP_DELETE)
+ t('/apisix/admin/consumers/foo2', ngx.HTTP_DELETE)
+
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "test",
+ "plugins": {
+ "basic-auth": {
+ "username": "test",
+ "password": "test"
+ }
+ }
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ local pwds = {}
+ table.insert(pwds, res.list[1].value.plugins["basic-auth"].password)
+ table.insert(pwds, res.list[2].value.plugins["basic-auth"].password)
+
+ ngx.say(json.encode(pwds))
+ }
+ }
+--- request
+GET /t
Review Comment:
Ditto
##########
t/node/data_encrypt.t:
##########
@@ -0,0 +1,581 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+# the sensitive data is encrypted in etcd, and it is normal to read it from the admin API
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+77+NmbYqNfN+oLm0aX5akg==
+
+
+
+=== TEST 2: enable basic auth plugin
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
Review Comment:
`GET /t` is set at the top of this file
##########
docs/en/latest/plugin-develop.md:
##########
@@ -293,6 +293,38 @@ function _M.check_schema(conf, schema_type)
end
```
+### encrypted storage fields
+
+Specify the parameters to be stored encrypted(Requires APISIX version > 3.0.1)
Review Comment:
```suggestion
Specify the parameters to be stored encrypted. (Requires APISIX version >= 3.1.0)
```
##########
docs/zh/latest/plugin-develop.md:
##########
@@ -273,6 +273,38 @@ function _M.check_schema(conf, schema_type)
end
```
+### 加密存储字段
+
+指定参数需要被加密存储（需要 APISIX 版本大于 3.0.1）
Review Comment:
```suggestion
指定参数需要被加密存储（需要 APISIX 版本不小于 3.1）
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1033198875
##########
apisix/consumer.lua:
##########
@@ -32,6 +33,23 @@ local lrucache = core.lrucache.new({
ttl = 300, count = 512
})
+
+local function decrypt_items(name, conf)
+ local schema = plugin.get(name)
+ local consumer_schema = schema.consumer_schema
+ if not consumer_schema then
+ return
+ end
+
+ for key, props in pairs(consumer_schema.properties) do
+ if props.type == "string" and props.encrypted then
+ local encrypted = apisix_ssl.aes_decrypt_pkey(conf[key], "global_data_encrypt")
+ conf[key] = encrypted
+ end
+ end
Review Comment:
The schema may have nested objects
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034221441
##########
apisix/admin/routes.lua:
##########
@@ -131,6 +133,9 @@ local function check_conf(id, conf, need_id)
if not ok then
return nil, {error_msg = err}
end
+ for name, conf in pairs(conf.plugins) do
Review Comment:
can we do this at lower level? it's not convenient to add this to every admin API,
##########
apisix/admin/routes.lua:
##########
@@ -131,6 +133,9 @@ local function check_conf(id, conf, need_id)
if not ok then
return nil, {error_msg = err}
end
+ for name, conf in pairs(conf.plugins) do
Review Comment:
can we do this at lower level? it's not convenient to add this to every admin API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034216088
##########
apisix/plugin.lua:
##########
@@ -901,7 +961,15 @@ _M.stream_check_schema = stream_check_schema
function _M.plugin_checker(item, schema_type)
if item.plugins then
- return check_schema(item.plugins, schema_type, true)
+ local ok, err = check_schema(item.plugins, schema_type, true)
+
+ if ok then
Review Comment:
would check `data_encryption == true` in `decrypt_conf` function, ref: https://github.com/apache/apisix/pull/8403/files/6fd1700dd37fc68d4b057d302229c06bf342eb24#diff-3936316fb5318f20b79ffe05648cf3d6b8c61e6987b51b52e2571c8a8e73a68bR867-R870
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035476389
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,65 @@ check_plugin_metadata = function(item)
end
+local function check_enable_and_get_plugin_schema(name, schema_type)
+ local plugin_schema = local_plugins_hash and local_plugins_hash[name]
+ local schema
+ if schema_type == core.schema.TYPE_CONSUMER then
+ schema = plugin_schema.consumer_schema
+ else
+ schema = plugin_schema.schema
+ end
+
+ return schema
+end
+
+
+local function decrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
+ if props.type == "string" and props.encrypted and conf[key] then
+ local encrypted, err = apisix_ssl.aes_decrypt_pkey(conf[key], "data_encrypt")
+ if not encrypted then
+ core.log.warn("failed to decrypt the conf of plugin [", name,
+ "] key [", key, "], err: ", err)
+ else
+ conf[key] = encrypted
+ end
+ end
+ end
+end
+_M.decrypt_conf = decrypt_conf
+
+
+local function encrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
Review Comment:
Anyof or oneof only restricts the existence of items, if an item has `encrypted = true` but does not exist in conf, it will not be encrypted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034585597
##########
apisix/admin/routes.lua:
##########
@@ -131,6 +133,9 @@ local function check_conf(id, conf, need_id)
if not ok then
return nil, {error_msg = err}
end
+ for name, conf in pairs(conf.plugins) do
Review Comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034247955
##########
t/plugin/clickhouse-logger.t:
##########
@@ -236,3 +236,235 @@ clickhouse headers: x-clickhouse-key:a
clickhouse headers: x-clickhouse-user:default
clickhouse headers: x-clickhouse-database:default
--- wait: 5
+
+
+
+=== TEST 7: password use data encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "clickhouse-logger": {
+ "user": "default",
+ "password": "abc123",
+ "database": "default",
+ "logtable": "t",
+ "endpoint_addr": "http://127.0.0.1:10420/clickhouse-logger/test",
+ "batch_max_size":1,
+ "inactive_timeout":1
+ }
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1982": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/opentracing"
+ }]]
+ )
+
+ ngx.sleep(0.5)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/routes/1',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["clickhouse-logger"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/routes/1'))
+ ngx.say(res.body.node.value.plugins["clickhouse-logger"].password)
+ }
+ }
+--- response_body
+abc123
+7ipXoKyiZZUAgf3WWNPI5A==
+--- error_log
+base64 decode ssl key failed. key[a]
+
+
+
+=== TEST 8: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /opentracing
+--- response_body
+opentracing
+--- error_log
+clickhouse body: INSERT INTO t FORMAT JSONEachRow
+clickhouse headers: x-clickhouse-key:abc123
+clickhouse headers: x-clickhouse-user:default
+clickhouse headers: x-clickhouse-database:default
+--- wait: 5
+
+
+
+=== TEST 9: POST and get lis
Review Comment:
```suggestion
=== TEST 9: POST and get list
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034684665
##########
docs/zh/latest/plugin-develop.md:
##########
@@ -273,6 +273,38 @@ function _M.check_schema(conf, schema_type)
end
```
+### 加密存储字段
+
+指定参数需要被加密存储（需要 APISIX 版本大于 3.0.1）
Review Comment:
done
##########
t/node/data_encrypt.t:
##########
@@ -0,0 +1,581 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034684194
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
+ end
+
+ return enable_data_encryption
+end
+
+
+local function get_plugin_schema(name, schema_type)
+ if not enable_gde() then
+ return false
Review Comment:
done
##########
apisix/ssl.lua:
##########
@@ -173,7 +218,11 @@ end
local function parse_pem_priv_key(sni, pkey)
core.log.debug("parsing priv key for sni: ", sni)
- local parsed, err = ngx_ssl.parse_pem_priv_key(aes_decrypt_pkey(pkey))
+ local key, err = aes_decrypt_pkey(pkey)
+ if not key then
+ core.log.error(err)
Review Comment:
done
##########
docs/en/latest/plugin-develop.md:
##########
@@ -293,6 +293,38 @@ function _M.check_schema(conf, schema_type)
end
```
+### encrypted storage fields
+
+Specify the parameters to be stored encrypted(Requires APISIX version > 3.0.1)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035437241
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,65 @@ check_plugin_metadata = function(item)
end
+local function check_enable_and_get_plugin_schema(name, schema_type)
+ local plugin_schema = local_plugins_hash and local_plugins_hash[name]
+ local schema
+ if schema_type == core.schema.TYPE_CONSUMER then
+ schema = plugin_schema.consumer_schema
+ else
+ schema = plugin_schema.schema
+ end
+
+ return schema
+end
+
+
+local function decrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
+ if props.type == "string" and props.encrypted and conf[key] then
+ local encrypted, err = apisix_ssl.aes_decrypt_pkey(conf[key], "data_encrypt")
+ if not encrypted then
+ core.log.warn("failed to decrypt the conf of plugin [", name,
+ "] key [", key, "], err: ", err)
+ else
+ conf[key] = encrypted
+ end
+ end
+ end
+end
+_M.decrypt_conf = decrypt_conf
+
+
+local function encrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
Review Comment:
I don't think it's needed and will finish it soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035497157
##########
t/node/data_encrypt.t:
##########
@@ -0,0 +1,581 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+# the sensitive data is encrypted in etcd, and it is normal to read it from the admin API
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+77+NmbYqNfN+oLm0aX5akg==
+
+
+
+=== TEST 2: enable basic auth plugin
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
+--- response_body
+passed
+
+
+
+=== TEST 3: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 4: multiple auth plugins work well
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ },
+ "key-auth": {
+ "key": "auth-one"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say("done")
+ }
+ }
+--- response_body
+done
+
+
+
+=== TEST 5: enable multiple auth plugins on route
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {},
+ "key-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
+--- response_body
+passed
+
+
+
+=== TEST 6: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+apikey: auth-one
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 7: disable data_encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: false
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+
+
+
+=== TEST 8: etcd store unencrypted password, enable data_encryption, decryption fails, use original password
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local core = require("apisix.core")
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res, err = core.etcd.set("/consumers/foo2", core.json.decode([[{
+ "username":"foo2",
+ "plugins":{
+ "basic-auth":{
+ "username":"foo2",
+ "password":"bar"
+ }
+ }
+ }]]))
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo2',
+ ngx.HTTP_GET
+ )
+ res = core.json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo2'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+ }
+ }
+--- response_body
+bar
+bar
+--- error_log
+failed to decrypt the conf of plugin [basic-auth] key [password], err: decrypt ssl key failed
+
+
+
+=== TEST 9: etcd stores both encrypted and unencrypted data
+# enable data_encryption, decryption of encrypted data succeeds
+# decryption of unencrypted data fails, make sure it works well
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local core = require("apisix.core")
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res, err = core.etcd.set("/consumers/foo2", core.json.decode([[{
+ "username":"foo2",
+ "plugins":{
+ "basic-auth":{
+ "username":"foo2",
+ "password":"bar"
+ },
+ "key-auth": {
+ "key": "vU/ZHVJw7b0XscDJ1Fhtig=="
+ }
+ }
+ }]]))
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo2',
+ ngx.HTTP_GET
+ )
+ res = core.json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+ ngx.say(res.value.plugins["key-auth"].key)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo2'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+ ngx.say(res.body.node.value.plugins["key-auth"].key)
+ }
+ }
+--- response_body
+bar
+auth-two
+bar
+vU/ZHVJw7b0XscDJ1Fhtig==
+--- error_log
+failed to decrypt the conf of plugin [basic-auth] key [password], err: decrypt ssl key failed
+
+
+
+=== TEST 10: verify, use the foo2 consumer
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+apikey: auth-two
+Authorization: Basic Zm9vMjpiYXI=
+--- response_body
+hello world
+
+
+
+=== TEST 11: keyring rotate, encrypt with edd1c9f0985e76a2
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- response_body
+passed
+
+
+
+=== TEST 12: keyring rotate, decrypt with qeddd145sfvddff3 would fail, but encrypt with edd1c9f0985e76a2 would success
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - qeddd145sfvddff3
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 13: search consumer list
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+
+ -- dletet exist consumers
+ t('/apisix/admin/consumers/foo', ngx.HTTP_DELETE)
+ t('/apisix/admin/consumers/foo2', ngx.HTTP_DELETE)
+
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "test",
+ "plugins": {
+ "basic-auth": {
+ "username": "test",
+ "password": "test"
+ }
+ }
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ local pwds = {}
+ table.insert(pwds, res.list[1].value.plugins["basic-auth"].password)
+ table.insert(pwds, res.list[2].value.plugins["basic-auth"].password)
+
+ ngx.say(json.encode(pwds))
+ }
+ }
+--- request
+GET /t
Review Comment:
my mistake, remove this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034220292
##########
docs/zh/latest/plugin-develop.md:
##########
@@ -273,6 +273,37 @@ function _M.check_schema(conf, schema_type)
end
```
+指定参数需要被加密存储（需要 APISIX 版本大于 3.0.1）
+
+有些插件需要将参数加密存储，比如 `basic-auth` 插件的 `password` 参数。这个插件需要在 `schema` 中指定哪些参数需要被加密存储。
+
+```lua
+password = { type = "string", encrypted = true },
+```
+
+通过在 `schema` 中指定 `encrypted = true`，可以将参数加密存储。APISIX 将提供以下功能：
+
+- 通过 `Admin API` 来新增和更新资源时，对于 `encrypted = true` 的参数，APISIX 会自动加密存储在 etcd 中
+- 通过 `Admin API` 来获取资源时，对于 `encrypted = true` 的参数，APISIX 会自动解密返回
+- APISIX 在使用 `encrypted = true` 的参数时，会自动解密
Review Comment:
can we delete one of them?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1033423399
##########
apisix/consumer.lua:
##########
@@ -32,6 +33,23 @@ local lrucache = core.lrucache.new({
ttl = 300, count = 512
})
+
+local function decrypt_items(name, conf)
+ local schema = plugin.get(name)
+ local consumer_schema = schema.consumer_schema
+ if not consumer_schema then
+ return
+ end
+
+ for key, props in pairs(consumer_schema.properties) do
+ if props.type == "string" and props.encrypted then
+ local encrypted = apisix_ssl.aes_decrypt_pkey(conf[key], "global_data_encrypt")
+ conf[key] = encrypted
+ end
+ end
Review Comment:
I know that since this PR does not include this case, we can optimize this point in the next PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034204468
##########
docs/zh/latest/plugin-develop.md:
##########
@@ -273,6 +273,37 @@ function _M.check_schema(conf, schema_type)
end
```
+指定参数需要被加密存储（需要 APISIX 版本大于 3.0.1）
+
+有些插件需要将参数加密存储，比如 `basic-auth` 插件的 `password` 参数。这个插件需要在 `schema` 中指定哪些参数需要被加密存储。
+
+```lua
+password = { type = "string", encrypted = true },
+```
+
+通过在 `schema` 中指定 `encrypted = true`，可以将参数加密存储。APISIX 将提供以下功能：
+
+- 通过 `Admin API` 来新增和更新资源时，对于 `encrypted = true` 的参数，APISIX 会自动加密存储在 etcd 中
+- 通过 `Admin API` 来获取资源时，对于 `encrypted = true` 的参数，APISIX 会自动解密返回
+- APISIX 在使用 `encrypted = true` 的参数时，会自动解密
Review Comment:
essentially, it's the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035569987
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
Review Comment:
1. if the user doesn't config `apisix.data_encryption.enable`, then `core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")` would get the default value `false` from `conf/config-default.yaml`
2. `enable_data_encryption` would be set to `false` as module var.
3. in the next time, `enable_data_encryption == nil` would get false
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander merged pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
spacewander merged PR #8403:
URL: https://github.com/apache/apisix/pull/8403
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034193462
##########
docs/zh/latest/plugin-develop.md:
##########
@@ -273,6 +273,37 @@ function _M.check_schema(conf, schema_type)
end
```
+指定参数需要被加密存储（需要 APISIX 版本大于 3.0.1）
+
+有些插件需要将参数加密存储，比如 `basic-auth` 插件的 `password` 参数。这个插件需要在 `schema` 中指定哪些参数需要被加密存储。
+
+```lua
+password = { type = "string", encrypted = true },
+```
+
+通过在 `schema` 中指定 `encrypted = true`，可以将参数加密存储。APISIX 将提供以下功能：
+
+- 通过 `Admin API` 来新增和更新资源时，对于 `encrypted = true` 的参数，APISIX 会自动加密存储在 etcd 中
+- 通过 `Admin API` 来获取资源时，对于 `encrypted = true` 的参数，APISIX 会自动解密返回
+- APISIX 在使用 `encrypted = true` 的参数时，会自动解密
Review Comment:
The two line means the same thing?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034556937
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,65 @@ check_plugin_metadata = function(item)
end
+local function check_enable_and_get_plugin_schema(name, schema_type)
+ local plugin_schema = local_plugins_hash and local_plugins_hash[name]
+ local schema
+ if schema_type == core.schema.TYPE_CONSUMER then
+ schema = plugin_schema.consumer_schema
+ else
+ schema = plugin_schema.schema
+ end
+
+ return schema
+end
+
+
+local function decrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
+ if props.type == "string" and props.encrypted and conf[key] then
+ local encrypted, err = apisix_ssl.aes_decrypt_pkey(conf[key], "data_encrypt")
+ if not encrypted then
+ core.log.warn("failed to decrypt the conf of plugin [", name,
+ "] key [", key, "], err: ", err)
+ else
+ conf[key] = encrypted
+ end
+ end
+ end
+end
+_M.decrypt_conf = decrypt_conf
+
+
+local function encrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
Review Comment:
I know that since this PR does not include this case, we can optimize this point in the next PR.
##########
apisix/plugins/basic-auth.lua:
##########
@@ -39,7 +38,7 @@ local consumer_schema = {
title = "work with consumer object",
properties = {
username = { type = "string" },
- password = { type = "string" },
+ password = { type = "string", encrypted = true },
Review Comment:
added
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035430569
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,65 @@ check_plugin_metadata = function(item)
end
+local function check_enable_and_get_plugin_schema(name, schema_type)
+ local plugin_schema = local_plugins_hash and local_plugins_hash[name]
+ local schema
+ if schema_type == core.schema.TYPE_CONSUMER then
+ schema = plugin_schema.consumer_schema
+ else
+ schema = plugin_schema.schema
+ end
+
+ return schema
+end
+
+
+local function decrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
+ if props.type == "string" and props.encrypted and conf[key] then
+ local encrypted, err = apisix_ssl.aes_decrypt_pkey(conf[key], "data_encrypt")
+ if not encrypted then
+ core.log.warn("failed to decrypt the conf of plugin [", name,
+ "] key [", key, "], err: ", err)
+ else
+ conf[key] = encrypted
+ end
+ end
+ end
+end
+_M.decrypt_conf = decrypt_conf
+
+
+local function encrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
Review Comment:
It's better to notice this point at doc ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
soulbird commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035498053
##########
t/node/data_encrypt2.t:
##########
@@ -0,0 +1,731 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!defined $block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ my $http_config = $block->http_config // <<_EOC_;
+ server {
+ listen 10420;
+ location /clickhouse-logger/test {
+ content_by_lua_block {
+ ngx.req.read_body()
+ local data = ngx.req.get_body_data()
+ local headers = ngx.req.get_headers()
+ ngx.log(ngx.WARN, "clickhouse body: ", data)
+ for k, v in pairs(headers) do
+ ngx.log(ngx.WARN, "clickhouse headers: " .. k .. ":" .. v)
+ end
+ ngx.say("ok")
+ }
+ }
+ location /clickhouse-logger/test1 {
+ content_by_lua_block {
+ ngx.req.read_body()
+ local data = ngx.req.get_body_data()
+ local headers = ngx.req.get_headers()
+ ngx.log(ngx.WARN, "clickhouse body: ", data)
+ for k, v in pairs(headers) do
+ ngx.log(ngx.WARN, "clickhouse headers: " .. k .. ":" .. v)
+ end
+ ngx.say("ok")
+ }
+ }
+ }
+_EOC_
Review Comment:
Can it be moved into t/lib? It is the same as in clickhouse-logger.t ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035438382
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,65 @@ check_plugin_metadata = function(item)
end
+local function check_enable_and_get_plugin_schema(name, schema_type)
+ local plugin_schema = local_plugins_hash and local_plugins_hash[name]
+ local schema
+ if schema_type == core.schema.TYPE_CONSUMER then
+ schema = plugin_schema.consumer_schema
+ else
+ schema = plugin_schema.schema
+ end
+
+ return schema
+end
+
+
+local function decrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
+ if props.type == "string" and props.encrypted and conf[key] then
+ local encrypted, err = apisix_ssl.aes_decrypt_pkey(conf[key], "data_encrypt")
+ if not encrypted then
+ core.log.warn("failed to decrypt the conf of plugin [", name,
+ "] key [", key, "], err: ", err)
+ else
+ conf[key] = encrypted
+ end
+ end
+ end
+end
+_M.decrypt_conf = decrypt_conf
+
+
+local function encrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
Review Comment:
It's a bit complicated, like 'anyof', 'oneof' need to be considered.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035475287
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
Review Comment:
The cache won't take effect when `core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")` is nil. is this by design?
##########
apisix/admin/utils.lua:
##########
@@ -78,4 +80,28 @@ function _M.fix_count(body, id)
end
+function _M.decrypt_params(decrypt_func, body, schema_type)
+ -- list
+ if body.list and #body.list > 0 then
Review Comment:
For the same reason, we can skip the `and #body.list > 0` too.
##########
t/node/data_encrypt.t:
##########
@@ -0,0 +1,581 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+no_shuffle();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: sanity
+# the sensitive data is encrypted in etcd, and it is normal to read it from the admin API
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+77+NmbYqNfN+oLm0aX5akg==
+
+
+
+=== TEST 2: enable basic auth plugin
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
+--- response_body
+passed
+
+
+
+=== TEST 3: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 4: multiple auth plugins work well
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ },
+ "key-auth": {
+ "key": "auth-one"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+ local code, message, res = t('/apisix/admin/consumers/foo',
+ ngx.HTTP_GET
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say("done")
+ }
+ }
+--- response_body
+done
+
+
+
+=== TEST 5: enable multiple auth plugins on route
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {},
+ "key-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- request
+GET /t
+--- response_body
+passed
+
+
+
+=== TEST 6: verify
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+apikey: auth-one
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 7: disable data_encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: false
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+
+ }
+ }
+--- response_body
+bar
+
+
+
+=== TEST 8: etcd store unencrypted password, enable data_encryption, decryption fails, use original password
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local core = require("apisix.core")
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res, err = core.etcd.set("/consumers/foo2", core.json.decode([[{
+ "username":"foo2",
+ "plugins":{
+ "basic-auth":{
+ "username":"foo2",
+ "password":"bar"
+ }
+ }
+ }]]))
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo2',
+ ngx.HTTP_GET
+ )
+ res = core.json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo2'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+ }
+ }
+--- response_body
+bar
+bar
+--- error_log
+failed to decrypt the conf of plugin [basic-auth] key [password], err: decrypt ssl key failed
+
+
+
+=== TEST 9: etcd stores both encrypted and unencrypted data
+# enable data_encryption, decryption of encrypted data succeeds
+# decryption of unencrypted data fails, make sure it works well
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local t = require("lib.test_admin").test
+ local core = require("apisix.core")
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res, err = core.etcd.set("/consumers/foo2", core.json.decode([[{
+ "username":"foo2",
+ "plugins":{
+ "basic-auth":{
+ "username":"foo2",
+ "password":"bar"
+ },
+ "key-auth": {
+ "key": "vU/ZHVJw7b0XscDJ1Fhtig=="
+ }
+ }
+ }]]))
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foo2',
+ ngx.HTTP_GET
+ )
+ res = core.json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["basic-auth"].password)
+ ngx.say(res.value.plugins["key-auth"].key)
+
+ -- get plugin conf from etcd, password is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foo2'))
+ ngx.say(res.body.node.value.plugins["basic-auth"].password)
+ ngx.say(res.body.node.value.plugins["key-auth"].key)
+ }
+ }
+--- response_body
+bar
+auth-two
+bar
+vU/ZHVJw7b0XscDJ1Fhtig==
+--- error_log
+failed to decrypt the conf of plugin [basic-auth] key [password], err: decrypt ssl key failed
+
+
+
+=== TEST 10: verify, use the foo2 consumer
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+apikey: auth-two
+Authorization: Basic Zm9vMjpiYXI=
+--- response_body
+hello world
+
+
+
+=== TEST 11: keyring rotate, encrypt with edd1c9f0985e76a2
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "basic-auth": {}
+ },
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "uri": "/hello"
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ end
+ ngx.say(body)
+ }
+ }
+--- response_body
+passed
+
+
+
+=== TEST 12: keyring rotate, decrypt with qeddd145sfvddff3 would fail, but encrypt with edd1c9f0985e76a2 would success
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - qeddd145sfvddff3
+ - edd1c9f0985e76a2
+--- request
+GET /hello
+--- more_headers
+Authorization: Basic Zm9vOmJhcg==
+--- response_body
+hello world
+
+
+
+=== TEST 13: search consumer list
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+
+ -- dletet exist consumers
+ t('/apisix/admin/consumers/foo', ngx.HTTP_DELETE)
+ t('/apisix/admin/consumers/foo2', ngx.HTTP_DELETE)
+
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foo",
+ "plugins": {
+ "basic-auth": {
+ "username": "foo",
+ "password": "bar"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/consumers',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "test",
+ "plugins": {
+ "basic-auth": {
+ "username": "test",
+ "password": "test"
+ }
+ }
+ }]]
+ )
+
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, password is decrypted
+ local code, message, res = t('/apisix/admin/consumers',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ local pwds = {}
+ table.insert(pwds, res.list[1].value.plugins["basic-auth"].password)
+ table.insert(pwds, res.list[2].value.plugins["basic-auth"].password)
+
+ ngx.say(json.encode(pwds))
+ }
+ }
+--- request
+GET /t
Review Comment:
@tzssangglass
Err. The `GET /t` here is not removed yet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
soulbird commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035489890
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035503165
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
Review Comment:
from #8407
```
apisix:
data_encryption:
enable: false # If not set, the default value is `false`.
```
if `apisix.data_encryption.enable` is nil, it is the same as not set `apisix.data_encryption.enable`, then `_M.enable_data_encryption` would be `false`.
`_M.enable_data_encryption` is used by `if method == "get" and plugin.enable_data_encryption then` in `apisix/admin/init.lua`, it won't into `utils.decrypt_params`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034556540
##########
conf/config-default.yaml:
##########
@@ -123,6 +123,13 @@ apisix:
# ip: 127.0.0.1
# port: 9090
disable_sync_configuration_during_start: false # safe exit. Remove this once the feature is stable
+ data_encryption: # use `encrypted = {fields}` in plugin schema to enable encryption
+ enable: false # if not set, the default value is `false`.
+ keyring:
+ - edd1c9f0985e76a2 # If not set, will save origin value into etcd.
Review Comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034204572
##########
apisix/plugin.lua:
##########
@@ -901,7 +961,15 @@ _M.stream_check_schema = stream_check_schema
function _M.plugin_checker(item, schema_type)
if item.plugins then
- return check_schema(item.plugins, schema_type, true)
+ local ok, err = check_schema(item.plugins, schema_type, true)
+
+ if ok then
Review Comment:
Add judge `data_encryption == true` here ? if `data_encryption == false`, no need to iterate the plugin loop
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
membphis commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034243576
##########
conf/config-default.yaml:
##########
@@ -123,6 +123,13 @@ apisix:
# ip: 127.0.0.1
# port: 9090
disable_sync_configuration_during_start: false # safe exit. Remove this once the feature is stable
+ data_encryption: # use `encrypted = {fields}` in plugin schema to enable encryption
+ enable: false # if not set, the default value is `false`.
+ keyring:
+ - edd1c9f0985e76a2 # If not set, will save origin value into etcd.
Review Comment:
we should use a different key
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034683801
##########
apisix/admin/utils.lua:
##########
@@ -78,4 +80,29 @@ function _M.fix_count(body, id)
end
+function _M.decrypt_params(decrypt_func, body, schema_type)
+ -- list
+ if body.list and #body.list > 0 then
+ for _, route in ipairs(body.list) do
+ if route.value and route.value.plugins
+ and core.table.nkeys(route.value.plugins) > 0 then
Review Comment:
done
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
+ end
+
+ return enable_data_encryption
+end
+
+
+local function get_plugin_schema(name, schema_type)
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
spacewander commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035550283
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,71 @@ check_plugin_metadata = function(item)
end
+local enable_data_encryption
+local function enable_gde()
+ if enable_data_encryption == nil then
+ enable_data_encryption =
+ core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ _M.enable_data_encryption = enable_data_encryption
Review Comment:
@tzssangglass
`enable_data_encryption` would be nil in this case? So `enable_data_encryption == nil` will always get a true result and this function doesn't provide a cache if the user doesn't provide a configuration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1035497050
##########
apisix/admin/utils.lua:
##########
@@ -78,4 +80,28 @@ function _M.fix_count(body, id)
end
+function _M.decrypt_params(decrypt_func, body, schema_type)
+ -- list
+ if body.list and #body.list > 0 then
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] monkeyDluffy6017 commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
monkeyDluffy6017 commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034217388
##########
apisix/plugin.lua:
##########
@@ -901,7 +961,15 @@ _M.stream_check_schema = stream_check_schema
function _M.plugin_checker(item, schema_type)
if item.plugins then
- return check_schema(item.plugins, schema_type, true)
+ local ok, err = check_schema(item.plugins, schema_type, true)
+
+ if ok then
Review Comment:
But still need to iterate the loop below, we can skip the iteration
```
for name, conf in pairs(item.plugins) do
decrypt_conf(name, conf, schema_type)
end
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
soulbird commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034364481
##########
apisix/plugin.lua:
##########
@@ -849,6 +850,65 @@ check_plugin_metadata = function(item)
end
+local function check_enable_and_get_plugin_schema(name, schema_type)
+ local plugin_schema = local_plugins_hash and local_plugins_hash[name]
+ local schema
+ if schema_type == core.schema.TYPE_CONSUMER then
+ schema = plugin_schema.consumer_schema
+ else
+ schema = plugin_schema.schema
+ end
+
+ return schema
+end
+
+
+local function decrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
+ if props.type == "string" and props.encrypted and conf[key] then
+ local encrypted, err = apisix_ssl.aes_decrypt_pkey(conf[key], "data_encrypt")
+ if not encrypted then
+ core.log.warn("failed to decrypt the conf of plugin [", name,
+ "] key [", key, "], err: ", err)
+ else
+ conf[key] = encrypted
+ end
+ end
+ end
+end
+_M.decrypt_conf = decrypt_conf
+
+
+local function encrypt_conf(name, conf, schema_type)
+ local enable = core.table.try_read_attr(local_conf, "apisix", "data_encryption", "enable")
+ if not enable then
+ return conf
+ end
+
+ local schema = check_enable_and_get_plugin_schema(name, schema_type)
+ if not schema then
+ return
+ end
+
+ for key, props in pairs(schema.properties) do
Review Comment:
Don't we consider the case of configuration nesting here?
##########
apisix/plugins/basic-auth.lua:
##########
@@ -39,7 +38,7 @@ local consumer_schema = {
title = "work with consumer object",
properties = {
username = { type = "string" },
- password = { type = "string" },
+ password = { type = "string", encrypted = true },
Review Comment:
We need to adjust the documentation of the corresponding plugin for this point.
##########
t/node/consumer-group.t:
##########
@@ -310,3 +310,140 @@ world
}
--- response_body
bar
+
+
+
+=== TEST 5: data encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local etcd = require("apisix.core.etcd")
+ local code, body = t('/apisix/admin/consumer_groups/company_a',
Review Comment:
Why use consumer_groups?
##########
t/node/consumer-group.t:
##########
@@ -310,3 +310,140 @@ world
}
--- response_body
bar
+
+
+
+=== TEST 5: data encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local etcd = require("apisix.core.etcd")
+ local code, body = t('/apisix/admin/consumer_groups/company_a',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "limit-count": {
+ "count": 2,
+ "time_window": 60,
+ "rejected_code": 503,
+ "key": "remote_addr"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/consumers/foobar',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foobar",
+ "plugins": {
+ "key-auth": {
+ "key": "auth-two"
+ }
+ },
+ "group_id": "company_a"
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, key is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foobar',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["key-auth"].key)
+
+ -- get plugin conf from etcd, key is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foobar'))
+ ngx.say(res.body.node.value.plugins["key-auth"].key)
+ }
+ }
+--- response_body
+auth-two
+vU/ZHVJw7b0XscDJ1Fhtig==
+
+
+
+=== TEST 6: verify data encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require "t.toolkit.json"
+ local t = require("lib.test_admin").test
+ local code, err = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "uri": "/hello",
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "plugins": {
+ "key-auth": {}
+ }
+ }]]
+ )
+ if code > 300 then
+ ngx.log(ngx.ERR, err)
+ return
+ end
+ ngx.sleep(0.1)
+
+ local http = require "resty.http"
+ local uri = "http://127.0.0.1:" .. ngx.var.server_port
+ .. "/hello"
+ local ress = {}
+ for i = 1, 3 do
Review Comment:
Why not use `pipelined_requests`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034569648
##########
t/node/consumer-group.t:
##########
@@ -310,3 +310,140 @@ world
}
--- response_body
bar
+
+
+
+=== TEST 5: data encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local etcd = require("apisix.core.etcd")
+ local code, body = t('/apisix/admin/consumer_groups/company_a',
+ ngx.HTTP_PUT,
+ [[{
+ "plugins": {
+ "limit-count": {
+ "count": 2,
+ "time_window": 60,
+ "rejected_code": 503,
+ "key": "remote_addr"
+ }
+ }
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+
+ ngx.sleep(0.1)
+
+ local code, body = t('/apisix/admin/consumers/foobar',
+ ngx.HTTP_PUT,
+ [[{
+ "username": "foobar",
+ "plugins": {
+ "key-auth": {
+ "key": "auth-two"
+ }
+ },
+ "group_id": "company_a"
+ }]]
+ )
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(body)
+ return
+ end
+ ngx.sleep(0.1)
+
+ -- get plugin conf from admin api, key is decrypted
+ local code, message, res = t('/apisix/admin/consumers/foobar',
+ ngx.HTTP_GET
+ )
+ res = json.decode(res)
+ if code >= 300 then
+ ngx.status = code
+ ngx.say(message)
+ return
+ end
+
+ ngx.say(res.value.plugins["key-auth"].key)
+
+ -- get plugin conf from etcd, key is encrypted
+ local etcd = require("apisix.core.etcd")
+ local res = assert(etcd.get('/consumers/foobar'))
+ ngx.say(res.body.node.value.plugins["key-auth"].key)
+ }
+ }
+--- response_body
+auth-two
+vU/ZHVJw7b0XscDJ1Fhtig==
+
+
+
+=== TEST 6: verify data encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require "t.toolkit.json"
+ local t = require("lib.test_admin").test
+ local code, err = t('/apisix/admin/routes/1',
+ ngx.HTTP_PUT,
+ [[{
+ "uri": "/hello",
+ "upstream": {
+ "nodes": {
+ "127.0.0.1:1980": 1
+ },
+ "type": "roundrobin"
+ },
+ "plugins": {
+ "key-auth": {}
+ }
+ }]]
+ )
+ if code > 300 then
+ ngx.log(ngx.ERR, err)
+ return
+ end
+ ngx.sleep(0.1)
+
+ local http = require "resty.http"
+ local uri = "http://127.0.0.1:" .. ngx.var.server_port
+ .. "/hello"
+ local ress = {}
+ for i = 1, 3 do
Review Comment:
I need to pass auth headers
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a diff in pull request #8403: feat: support global data encryption of secret information
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on code in PR #8403:
URL: https://github.com/apache/apisix/pull/8403#discussion_r1034568983
##########
t/node/consumer-group.t:
##########
@@ -310,3 +310,140 @@ world
}
--- response_body
bar
+
+
+
+=== TEST 5: data encryption
+--- yaml_config
+apisix:
+ data_encryption:
+ enable: true
+ keyring:
+ - edd1c9f0985e76a2
+--- config
+ location /t {
+ content_by_lua_block {
+ local json = require("toolkit.json")
+ local t = require("lib.test_admin").test
+ local etcd = require("apisix.core.etcd")
+ local code, body = t('/apisix/admin/consumer_groups/company_a',
Review Comment:
make sure that data encryption works well with consumer groups
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org