You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/03/15 06:44:03 UTC
[GitHub] [apisix] tzssangglass opened a new pull request #6614: feat: support reading configuration form shdict(mvp)
tzssangglass opened a new pull request #6614:
URL: https://github.com/apache/apisix/pull/6614
### Description
<!-- Please include a summary of the change and which issue is fixed. -->
<!-- Please also include relevant motivation and context. -->
Fixes # (issue)
### Checklist
- [ ] I have explained the need for this PR and the problem it solves
- [ ] I have explained the changes or the new features added in this PR
- [ ] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [ ] I have verified that this change is backwards compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first)
<!--
Note
1. Mark PR as draft until its ready to be reviewed.
2. Always add/update tests for any changes unless you have a good reason.
3. Always update the documentation to reflect the changes made in the PR.
4. Make a new commit to resolve conversations instead of `push -f`.
5. To resolve merge conflicts, merge master instead of rebasing.
6. Use "request review" to notify the reviewer after you make changes.
7. Only a reviewer can mark a conversation as resolved.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r828789873
##########
File path: .github/workflows/build.yml
##########
@@ -29,7 +29,7 @@ jobs:
test_dir:
- t/plugin
- t/admin t/cli t/config-center-yaml t/control t/core t/debug t/discovery t/error_page t/misc
- - t/node t/router t/script t/stream-node t/utils t/wasm
+ - t/node t/router t/script t/stream-node t/utils t/wasm t/amesh-library
Review comment:
fixed
##########
File path: apisix/cli/ops.lua
##########
@@ -560,6 +560,7 @@ Please modify "admin_key" in conf/config.yaml .
end
sys_conf["wasm"] = yaml_conf.wasm
+ sys_conf["config_center"] = yaml_conf.apisix.config_center
Review comment:
rm this
##########
File path: apisix/cli/schema.lua
##########
@@ -28,7 +28,7 @@ local config_schema = {
apisix = {
properties = {
config_center = {
- enum = {"etcd", "yaml"},
+ enum = {"etcd", "yaml", "shdict"},
Review comment:
updated
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,121 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libamesh(lib_name)
+ ngx_timer_at(0, function(premature)
Review comment:
fixed
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,121 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libamesh(lib_name)
+ ngx_timer_at(0, function(premature)
+ if premature then
+ return
+ end
+
+ local ameshagent, tried_paths = load_shared_lib(lib_name)
+
+ if not ameshagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load '
+ .. lib_name
+ error("can not load Amesh library, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
+ local router_shd_cdata = ffi.cast("void*", router_zone)
+ ameshagent.initial(router_shd_cdata)
+ end)
+end
+
+
+
+function _M.init_worker()
+ local lib_name = "libamesh.so"
+
+ if process.type() == "privileged agent" then
+ load_libamesh(lib_name)
+ end
+
+ return true
+end
+
+
+function _M.new(key, opts)
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r829167348
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_shdict") then
Review comment:
yes, you are right. `config_xds` is fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r827748052
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,121 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* writeRoute);
Review comment:
we should use standard C style: `write_route `
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void CreateMock(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_ameshagent(lib_name)
+ ngx_timer_at(0, function(premature)
+ if premature then
+ return
+ end
+
+ local ameshagent, tried_paths = load_shared_lib(lib_name)
+
+ if not ameshagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load '
+ .. lib_name
+ error("can not load amesh agent, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
Review comment:
ok, got it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tao12345666333 commented on pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#issuecomment-1072028883
> So I think using the name xds is specific enough for this feature.
OK so we can keep this naming for now and let's move forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] leslie-tsang commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
leslie-tsang commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r831740941
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local route_config = ngx.shared["xds-route-config"]
Review comment:
```suggestion
local route_config = ngx.shared["xds-route-config"]
```
##########
File path: t/APISIX.pm
##########
@@ -509,6 +509,7 @@ _EOC_
lua_shared_dict ext-plugin 1m;
lua_shared_dict kubernetes 1m;
lua_shared_dict tars 1m;
+ lua_shared_dict xds-route-config 1m;
Review comment:
Should we sync the setting in `apisix/cli/ngx_tpl.lua` ?
Change `1m` to `10m`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r830778067
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["xds-route-config"]
Review comment:
There are still using `router` in some places.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r829167348
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_shdict") then
Review comment:
yes, you are right. `config_xds` is fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] shuaijinchao commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
shuaijinchao commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r829232087
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,119 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+local ngx_re_match = ngx.re.match
+local ngx_re_gmatch = ngx.re.gmatch
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ local iter, err = ngx_re_gmatch(cpath, "[^;]+", "jo")
+ if not iter then
+ error("failed to gmatch: " .. err)
+ end
+
+ while true do
+ local it = iter()
+ local fpath
+ fpath, err = ngx_re_match(it[0], "(.*/)", "jo")
+ if err then
+ error("failed to match: " .. err)
+ end
+ local spath = fpath[0] .. lib_name
+
+ local f = io_open(spath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(spath)
+ end
+ tried_paths[i] = spath
+ i = i + 1
+
+ if not it then
+ break
+ end
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libxds(lib_name)
+ local xdsagent, tried_paths = load_shared_lib(lib_name)
+
+ if not xdsagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load ' .. lib_name
+ error("can not load Amesh library, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
+ local router_shd_cdata = ffi.cast("void*", router_zone)
+ xdsagent.initial(router_shd_cdata)
+end
+
+
+
+function _M.init_worker()
+ local lib_name = "libxds.so"
Review comment:
Put it at the module level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r830886792
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["xds-route-config"]
Review comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r829651863
##########
File path: t/xds-library/config_xds.t
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+use Cwd qw(cwd);
+my $apisix_home = $ENV{APISIX_HOME} || cwd();
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+
+ my $lua_deps_path = $block->lua_deps_path // <<_EOC_;
+ lua_package_path "$apisix_home/?.lua;$apisix_home/?/init.lua;$apisix_home/deps/share/lua/5.1/?/init.lua;$apisix_home/deps/share/lua/5.1/?.lua;$apisix_home/apisix/?.lua;$apisix_home/t/?.lua;;";
+ lua_package_cpath "$apisix_home/?.so;$apisix_home/t/xds-library/?.so;$apisix_home/deps/lib/lua/5.1/?.so;$apisix_home/deps/lib64/lua/5.1/?.so;;";
+_EOC_
+
+ $block->set_value("lua_deps_path", $lua_deps_path);
+
+ my $extra_init_by_lua = <<_EOC_;
+ --
+ local config_xds = require("apisix.core.config_xds")
+
+ local inject = function(mod, name)
+ local old_f = mod[name]
+ mod[name] = function (...)
+ ngx.log(ngx.WARN, "config_xds run ", name)
+ return { true }
+ end
+ end
+
+ inject(config_xds, "new")
+
+_EOC_
+
+ $block->set_value("extra_init_by_lua", $extra_init_by_lua);
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: load Amesh library so successfully
+--- yaml_config
Review comment:
Let's set the yaml_config in the file level?
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_xds") then
Review comment:
We can call it like:
https://github.com/apache/apisix/blob/9d450d7fe3169a77727df28696d083809b93977a/apisix/init.lua#L83
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r829651863
##########
File path: t/xds-library/config_xds.t
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+use Cwd qw(cwd);
+my $apisix_home = $ENV{APISIX_HOME} || cwd();
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+
+ my $lua_deps_path = $block->lua_deps_path // <<_EOC_;
+ lua_package_path "$apisix_home/?.lua;$apisix_home/?/init.lua;$apisix_home/deps/share/lua/5.1/?/init.lua;$apisix_home/deps/share/lua/5.1/?.lua;$apisix_home/apisix/?.lua;$apisix_home/t/?.lua;;";
+ lua_package_cpath "$apisix_home/?.so;$apisix_home/t/xds-library/?.so;$apisix_home/deps/lib/lua/5.1/?.so;$apisix_home/deps/lib64/lua/5.1/?.so;;";
+_EOC_
+
+ $block->set_value("lua_deps_path", $lua_deps_path);
+
+ my $extra_init_by_lua = <<_EOC_;
+ --
+ local config_xds = require("apisix.core.config_xds")
+
+ local inject = function(mod, name)
+ local old_f = mod[name]
+ mod[name] = function (...)
+ ngx.log(ngx.WARN, "config_xds run ", name)
+ return { true }
+ end
+ end
+
+ inject(config_xds, "new")
+
+_EOC_
+
+ $block->set_value("extra_init_by_lua", $extra_init_by_lua);
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: load Amesh library so successfully
+--- yaml_config
Review comment:
Let's set the yaml_config in the file level?
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_xds") then
Review comment:
We can call it like:
https://github.com/apache/apisix/blob/9d450d7fe3169a77727df28696d083809b93977a/apisix/init.lua#L83
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r826962898
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void CreateMock(void* writeRoute);
Review comment:
`init()` is the name of a reserved function of go, with special meaning, I replaced it with `initial()`
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r828711976
##########
File path: apisix/cli/ops.lua
##########
@@ -560,6 +560,7 @@ Please modify "admin_key" in conf/config.yaml .
end
sys_conf["wasm"] = yaml_conf.wasm
+ sys_conf["config_center"] = yaml_conf.apisix.config_center
Review comment:
Handled in https://github.com/apache/apisix/blob/15517feb55a0c5e87c5fdb3b91d647ee0ca1c96f/apisix/cli/ops.lua#L555
##########
File path: apisix/cli/schema.lua
##########
@@ -28,7 +28,7 @@ local config_schema = {
apisix = {
properties = {
config_center = {
- enum = {"etcd", "yaml"},
+ enum = {"etcd", "yaml", "shdict"},
Review comment:
```suggestion
enum = {"etcd", "yaml", "xds"},
```
would be better?
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,121 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libamesh(lib_name)
+ ngx_timer_at(0, function(premature)
Review comment:
Why use the timer?
##########
File path: .github/workflows/build.yml
##########
@@ -29,7 +29,7 @@ jobs:
test_dir:
- t/plugin
- t/admin t/cli t/config-center-yaml t/control t/core t/debug t/discovery t/error_page t/misc
- - t/node t/router t/script t/stream-node t/utils t/wasm
+ - t/node t/router t/script t/stream-node t/utils t/wasm t/amesh-library
Review comment:
Should be sorted in order
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,121 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libamesh(lib_name)
+ ngx_timer_at(0, function(premature)
+ if premature then
+ return
+ end
+
+ local ameshagent, tried_paths = load_shared_lib(lib_name)
+
+ if not ameshagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load '
+ .. lib_name
+ error("can not load Amesh library, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
+ local router_shd_cdata = ffi.cast("void*", router_zone)
+ ameshagent.initial(router_shd_cdata)
+ end)
+end
+
+
+
+function _M.init_worker()
+ local lib_name = "libamesh.so"
+
+ if process.type() == "privileged agent" then
+ load_libamesh(lib_name)
+ end
+
+ return true
+end
+
+
+function _M.new(key, opts)
Review comment:
We can use injection for test?
https://github.com/apache/apisix/blob/15517feb55a0c5e87c5fdb3b91d647ee0ca1c96f/t/plugin/skywalking.t#L32-L50
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r828921285
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_shdict") then
Review comment:
another option, we can use `amesh`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tao12345666333 commented on pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tao12345666333 commented on pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#issuecomment-1072028883
> So I think using the name xds is specific enough for this feature.
OK so we can keep this naming for now and let's move forward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r830611241
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
Review comment:
It is not configure for router, but for route.
##########
File path: apisix/cli/ngx_tpl.lua
##########
@@ -235,6 +235,10 @@ http {
lua_shared_dict ext-plugin {* http.lua_shared_dict["ext-plugin"] *}; # cache for ext-plugin
{% end %}
+ {% if config_center == "xds" then %}
+ lua_shared_dict router-config 10m;
Review comment:
```suggestion
lua_shared_dict xds-route-config 10m;
```
would that be better?
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+local ngx_re_match = ngx.re.match
+local ngx_re_gmatch = ngx.re.gmatch
+
+local xds_lib_name = "libxds.so"
+
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ local iter, err = ngx_re_gmatch(cpath, "[^;]+", "jo")
+ if not iter then
+ error("failed to gmatch: " .. err)
+ end
+
+ while true do
+ local it = iter()
+ local fpath
+ fpath, err = ngx_re_match(it[0], "(.*/)", "jo")
+ if err then
+ error("failed to match: " .. err)
+ end
+ local spath = fpath[0] .. lib_name
+
+ local f = io_open(spath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(spath)
+ end
+ tried_paths[i] = spath
+ i = i + 1
+
+ if not it then
+ break
+ end
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libxds(lib_name)
+ local xdsagent, tried_paths = load_shared_lib(lib_name)
+
+ if not xdsagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load ' .. lib_name
+ error("can not load Amesh library, tried paths: " ..
Review comment:
```suggestion
error("can not load xds library, tried paths: " ..
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander merged pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander merged pull request #6614:
URL: https://github.com/apache/apisix/pull/6614
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r828972153
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,108 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
Review comment:
updated
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,108 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
Review comment:
updated
##########
File path: .github/workflows/build.yml
##########
@@ -90,6 +90,11 @@ jobs:
sudo dpkg -i tinygo_${TINYGO_VER}_amd64.deb
cd t/wasm && find . -type f -name "*.go" | xargs -Ip tinygo build -o p.wasm -scheduler=none -target=wasi p
+ - name: Build Amesh library
Review comment:
updated
##########
File path: .github/workflows/build.yml
##########
@@ -29,7 +29,7 @@ jobs:
test_dir:
- t/plugin
- t/admin t/cli t/config-center-yaml t/control t/core t/debug t/discovery t/error_page t/misc
- - t/node t/router t/script t/stream-node t/utils t/wasm
+ - t/amesh-library t/node t/router t/script t/stream-node t/utils t/wasm
Review comment:
fixed, use `xds`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r829251430
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,119 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+local ngx_re_match = ngx.re.match
+local ngx_re_gmatch = ngx.re.gmatch
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ local iter, err = ngx_re_gmatch(cpath, "[^;]+", "jo")
+ if not iter then
+ error("failed to gmatch: " .. err)
+ end
+
+ while true do
+ local it = iter()
+ local fpath
+ fpath, err = ngx_re_match(it[0], "(.*/)", "jo")
+ if err then
+ error("failed to match: " .. err)
+ end
+ local spath = fpath[0] .. lib_name
+
+ local f = io_open(spath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(spath)
+ end
+ tried_paths[i] = spath
+ i = i + 1
+
+ if not it then
+ break
+ end
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libxds(lib_name)
+ local xdsagent, tried_paths = load_shared_lib(lib_name)
+
+ if not xdsagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load ' .. lib_name
+ error("can not load Amesh library, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
+ local router_shd_cdata = ffi.cast("void*", router_zone)
+ xdsagent.initial(router_shd_cdata)
+end
+
+
+
+function _M.init_worker()
+ local lib_name = "libxds.so"
Review comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r830045579
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_xds") then
Review comment:
updated
##########
File path: t/xds-library/config_xds.t
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+use Cwd qw(cwd);
+my $apisix_home = $ENV{APISIX_HOME} || cwd();
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+
+ my $lua_deps_path = $block->lua_deps_path // <<_EOC_;
+ lua_package_path "$apisix_home/?.lua;$apisix_home/?/init.lua;$apisix_home/deps/share/lua/5.1/?/init.lua;$apisix_home/deps/share/lua/5.1/?.lua;$apisix_home/apisix/?.lua;$apisix_home/t/?.lua;;";
+ lua_package_cpath "$apisix_home/?.so;$apisix_home/t/xds-library/?.so;$apisix_home/deps/lib/lua/5.1/?.so;$apisix_home/deps/lib64/lua/5.1/?.so;;";
+_EOC_
+
+ $block->set_value("lua_deps_path", $lua_deps_path);
+
+ my $extra_init_by_lua = <<_EOC_;
+ --
+ local config_xds = require("apisix.core.config_xds")
+
+ local inject = function(mod, name)
+ local old_f = mod[name]
+ mod[name] = function (...)
+ ngx.log(ngx.WARN, "config_xds run ", name)
+ return { true }
+ end
+ end
+
+ inject(config_xds, "new")
+
+_EOC_
+
+ $block->set_value("extra_init_by_lua", $extra_init_by_lua);
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: load Amesh library so successfully
+--- yaml_config
Review comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander commented on pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#issuecomment-1071945183
IMHO, `external` is too general to know what it does. As etcd is an external service, what is the difference between `etcd` and `external`?
Moreover, we don't promise any API spec about the xds exchange format and we might optimize it for xds in the future. So I think using the name xds is specific enough for this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r829251430
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,119 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+local ngx_re_match = ngx.re.match
+local ngx_re_gmatch = ngx.re.gmatch
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ local iter, err = ngx_re_gmatch(cpath, "[^;]+", "jo")
+ if not iter then
+ error("failed to gmatch: " .. err)
+ end
+
+ while true do
+ local it = iter()
+ local fpath
+ fpath, err = ngx_re_match(it[0], "(.*/)", "jo")
+ if err then
+ error("failed to match: " .. err)
+ end
+ local spath = fpath[0] .. lib_name
+
+ local f = io_open(spath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(spath)
+ end
+ tried_paths[i] = spath
+ i = i + 1
+
+ if not it then
+ break
+ end
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libxds(lib_name)
+ local xdsagent, tried_paths = load_shared_lib(lib_name)
+
+ if not xdsagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load ' .. lib_name
+ error("can not load Amesh library, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
+ local router_shd_cdata = ffi.cast("void*", router_zone)
+ xdsagent.initial(router_shd_cdata)
+end
+
+
+
+function _M.init_worker()
+ local lib_name = "libxds.so"
Review comment:
updated
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_xds") then
Review comment:
updated
##########
File path: t/xds-library/config_xds.t
##########
@@ -0,0 +1,104 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+use Cwd qw(cwd);
+my $apisix_home = $ENV{APISIX_HOME} || cwd();
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+
+ my $lua_deps_path = $block->lua_deps_path // <<_EOC_;
+ lua_package_path "$apisix_home/?.lua;$apisix_home/?/init.lua;$apisix_home/deps/share/lua/5.1/?/init.lua;$apisix_home/deps/share/lua/5.1/?.lua;$apisix_home/apisix/?.lua;$apisix_home/t/?.lua;;";
+ lua_package_cpath "$apisix_home/?.so;$apisix_home/t/xds-library/?.so;$apisix_home/deps/lib/lua/5.1/?.so;$apisix_home/deps/lib64/lua/5.1/?.so;;";
+_EOC_
+
+ $block->set_value("lua_deps_path", $lua_deps_path);
+
+ my $extra_init_by_lua = <<_EOC_;
+ --
+ local config_xds = require("apisix.core.config_xds")
+
+ local inject = function(mod, name)
+ local old_f = mod[name]
+ mod[name] = function (...)
+ ngx.log(ngx.WARN, "config_xds run ", name)
+ return { true }
+ end
+ end
+
+ inject(config_xds, "new")
+
+_EOC_
+
+ $block->set_value("extra_init_by_lua", $extra_init_by_lua);
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: load Amesh library so successfully
+--- yaml_config
Review comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r830710278
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
Review comment:
updated
##########
File path: apisix/cli/ngx_tpl.lua
##########
@@ -235,6 +235,10 @@ http {
lua_shared_dict ext-plugin {* http.lua_shared_dict["ext-plugin"] *}; # cache for ext-plugin
{% end %}
+ {% if config_center == "xds" then %}
+ lua_shared_dict router-config 10m;
Review comment:
updated
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+local ngx_re_match = ngx.re.match
+local ngx_re_gmatch = ngx.re.gmatch
+
+local xds_lib_name = "libxds.so"
+
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ local iter, err = ngx_re_gmatch(cpath, "[^;]+", "jo")
+ if not iter then
+ error("failed to gmatch: " .. err)
+ end
+
+ while true do
+ local it = iter()
+ local fpath
+ fpath, err = ngx_re_match(it[0], "(.*/)", "jo")
+ if err then
+ error("failed to match: " .. err)
+ end
+ local spath = fpath[0] .. lib_name
+
+ local f = io_open(spath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(spath)
+ end
+ tried_paths[i] = spath
+ i = i + 1
+
+ if not it then
+ break
+ end
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libxds(lib_name)
+ local xdsagent, tried_paths = load_shared_lib(lib_name)
+
+ if not xdsagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load ' .. lib_name
+ error("can not load Amesh library, tried paths: " ..
Review comment:
updated
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
membphis commented on pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#issuecomment-1072900692
we can merge this PR after fixed CI problems
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r830611241
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
Review comment:
It is not configuration for router, but for route.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r826964320
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void CreateMock(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_ameshagent(lib_name)
+ ngx_timer_at(0, function(premature)
+ if premature then
+ return
+ end
+
+ local ameshagent, tried_paths = load_shared_lib(lib_name)
+
+ if not ameshagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load '
+ .. lib_name
+ error("can not load amesh agent, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
+ local router_shd_cdata = ffi.cast("void*", router_zone)
+ ameshagent.CreateMock(router_shd_cdata)
+ end)
+end
+
+
+
+function _M.init_worker()
+ local lib_name = "libameshagent"
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] leslie-tsang commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
leslie-tsang commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r831843344
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local route_config = ngx.shared["xds-route-config"]
Review comment:
OK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
tokers commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r828923290
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_shdict") then
Review comment:
@membphis I don't think so, from the point of view of Apache APISIX, what is `amesh`? Why do we need to care about a commercial product?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r826967583
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void CreateMock(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_ameshagent(lib_name)
+ ngx_timer_at(0, function(premature)
+ if premature then
+ return
+ end
+
+ local ameshagent, tried_paths = load_shared_lib(lib_name)
+
+ if not ameshagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load '
+ .. lib_name
+ error("can not load amesh agent, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
Review comment:
`router_config` type is `table`
`router_config[1]` type is `userdata`
`router_zone ` type is `cdata`
use `ngx_http_lua_ffi_shdict_udata_to_zone` here it is for passing pointer variables(`cdata`) to Amesh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
membphis commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r826632359
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void CreateMock(void* writeRoute);
Review comment:
change it to standard C style: `create_mock `
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void CreateMock(void* writeRoute);
Review comment:
we should call a `init()` function here
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
Review comment:
bad style, change to : `require("ffi")`
##########
File path: t/amesh-agent/config_shdict.t
##########
@@ -0,0 +1,84 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+use Cwd qw(cwd);
+my $apisix_home = $ENV{APISIX_HOME} || cwd();
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+
+ my $lua_deps_path = $block->lua_deps_path // <<_EOC_;
+ lua_package_path "$apisix_home/?.lua;$apisix_home/?/init.lua;$apisix_home/deps/share/lua/5.1/?/init.lua;$apisix_home/deps/share/lua/5.1/?.lua;$apisix_home/apisix/?.lua;$apisix_home/t/?.lua;;";
+ lua_package_cpath "$apisix_home/?.so;$apisix_home/t/amesh-agent/?.so;$apisix_home/deps/lib/lua/5.1/?.so;$apisix_home/deps/lib64/lua/5.1/?.so;;";
+_EOC_
+
+ $block->set_value("lua_deps_path", $lua_deps_path);
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: load amesh agent so successfully
+--- yaml_config
+apisix:
+ node_listen: 1984
+ config_center: shdict
+ enable_admin: false
+--- config
+ location /t {
+ content_by_lua_block {
+ ngx.say("ok")
+ }
+ }
+--- no_error_log eval
+qr/can not load amesh agent/
Review comment:
it should be a `Amesh library`
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void CreateMock(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_ameshagent(lib_name)
+ ngx_timer_at(0, function(premature)
+ if premature then
+ return
+ end
+
+ local ameshagent, tried_paths = load_shared_lib(lib_name)
+
+ if not ameshagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load '
+ .. lib_name
+ error("can not load amesh agent, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
Review comment:
why do we need to call this function?
##########
File path: apisix/core/config_shdict.lua
##########
@@ -0,0 +1,126 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_shdict
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require 'ffi'
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void CreateMock(void* writeRoute);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
+ fpath = fpath .. lib_name
+
+ local f = io_open(fpath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(fpath)
+ end
+ tried_paths[i] = fpath
+ i = i + 1
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_ameshagent(lib_name)
+ ngx_timer_at(0, function(premature)
+ if premature then
+ return
+ end
+
+ local ameshagent, tried_paths = load_shared_lib(lib_name)
+
+ if not ameshagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load '
+ .. lib_name
+ error("can not load amesh agent, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
+ local router_shd_cdata = ffi.cast("void*", router_zone)
+ ameshagent.CreateMock(router_shd_cdata)
+ end)
+end
+
+
+
+function _M.init_worker()
+ local lib_name = "libameshagent"
Review comment:
`libamesh` is fine
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] shuaijinchao commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
shuaijinchao commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r829232087
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,119 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+local ngx_re_match = ngx.re.match
+local ngx_re_gmatch = ngx.re.gmatch
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ local iter, err = ngx_re_gmatch(cpath, "[^;]+", "jo")
+ if not iter then
+ error("failed to gmatch: " .. err)
+ end
+
+ while true do
+ local it = iter()
+ local fpath
+ fpath, err = ngx_re_match(it[0], "(.*/)", "jo")
+ if err then
+ error("failed to match: " .. err)
+ end
+ local spath = fpath[0] .. lib_name
+
+ local f = io_open(spath)
+ if f ~= nil then
+ io_close(f)
+ return ffi.load(spath)
+ end
+ tried_paths[i] = spath
+ i = i + 1
+
+ if not it then
+ break
+ end
+ end
+
+ return nil, tried_paths
+end
+
+
+local function load_libxds(lib_name)
+ local xdsagent, tried_paths = load_shared_lib(lib_name)
+
+ if not xdsagent then
+ tried_paths[#tried_paths + 1] = 'tried above paths but can not load ' .. lib_name
+ error("can not load Amesh library, tried paths: " ..
+ table.concat(tried_paths, '\r\n', 1, #tried_paths))
+ end
+
+ local router_zone = C.ngx_http_lua_ffi_shdict_udata_to_zone(router_config[1])
+ local router_shd_cdata = ffi.cast("void*", router_zone)
+ xdsagent.initial(router_shd_cdata)
+end
+
+
+
+function _M.init_worker()
+ local lib_name = "libxds.so"
Review comment:
Put it at the module level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r831825854
##########
File path: t/APISIX.pm
##########
@@ -509,6 +509,7 @@ _EOC_
lua_shared_dict ext-plugin 1m;
lua_shared_dict kubernetes 1m;
lua_shared_dict tars 1m;
+ lua_shared_dict xds-route-config 1m;
Review comment:
It is enough in the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r826961829
##########
File path: t/amesh-agent/config_shdict.t
##########
@@ -0,0 +1,84 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+use t::APISIX 'no_plan';
+
+use Cwd qw(cwd);
+my $apisix_home = $ENV{APISIX_HOME} || cwd();
+
+repeat_each(1);
+no_long_string();
+no_root_location();
+log_level("info");
+
+add_block_preprocessor(sub {
+ my ($block) = @_;
+
+ if (!$block->request) {
+ $block->set_value("request", "GET /t");
+ }
+
+ if (!$block->no_error_log) {
+ $block->set_value("no_error_log", "[error]\n[alert]");
+ }
+
+ my $lua_deps_path = $block->lua_deps_path // <<_EOC_;
+ lua_package_path "$apisix_home/?.lua;$apisix_home/?/init.lua;$apisix_home/deps/share/lua/5.1/?/init.lua;$apisix_home/deps/share/lua/5.1/?.lua;$apisix_home/apisix/?.lua;$apisix_home/t/?.lua;;";
+ lua_package_cpath "$apisix_home/?.so;$apisix_home/t/amesh-agent/?.so;$apisix_home/deps/lib/lua/5.1/?.so;$apisix_home/deps/lib64/lua/5.1/?.so;;";
+_EOC_
+
+ $block->set_value("lua_deps_path", $lua_deps_path);
+});
+
+run_tests;
+
+__DATA__
+
+=== TEST 1: load amesh agent so successfully
+--- yaml_config
+apisix:
+ node_listen: 1984
+ config_center: shdict
+ enable_admin: false
+--- config
+ location /t {
+ content_by_lua_block {
+ ngx.say("ok")
+ }
+ }
+--- no_error_log eval
+qr/can not load amesh agent/
Review comment:
fixed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on a change in pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r831826866
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,120 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local io = io
+local io_open = io.open
+local io_close = io.close
+local package = package
+local new_tab = base.new_tab
+local ffi = require ("ffi")
+local C = ffi.C
+local route_config = ngx.shared["xds-route-config"]
Review comment:
I want to change it in the next PR😅, otherwise it will dismiss the previous approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] membphis removed a comment on pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
membphis removed a comment on pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#issuecomment-1072900692
we can merge this PR after fixed CI problems
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on a change in pull request #6614: feat: support reading configuration form shdict(mvp)
Posted by GitBox <gi...@apache.org>.
tokers commented on a change in pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#discussion_r828910189
##########
File path: .github/workflows/build.yml
##########
@@ -90,6 +90,11 @@ jobs:
sudo dpkg -i tinygo_${TINYGO_VER}_amd64.deb
cd t/wasm && find . -type f -name "*.go" | xargs -Ip tinygo build -o p.wasm -scheduler=none -target=wasi p
+ - name: Build Amesh library
Review comment:
Ditto.
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,108 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
Review comment:
Why not use `ngx.re.gmatch`?
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,108 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
+
+ local cpath = package.cpath
+ local tried_paths = new_tab(32, 0)
+ local i = 1
+
+ for k, _ in string_gmatch(cpath, "[^;]+") do
+ local fpath = string_match(k, "(.*/)")
Review comment:
What about `ngx.re.match`?
##########
File path: .github/workflows/build.yml
##########
@@ -29,7 +29,7 @@ jobs:
test_dir:
- t/plugin
- t/admin t/cli t/config-center-yaml t/control t/core t/debug t/discovery t/error_page t/misc
- - t/node t/router t/script t/stream-node t/utils t/wasm
+ - t/amesh-library t/node t/router t/script t/stream-node t/utils t/wasm
Review comment:
What is amesh?
##########
File path: apisix/init.lua
##########
@@ -110,6 +110,11 @@ function _M.http_init_worker()
end
require("apisix.balancer").init_worker()
load_balancer = require("apisix.balancer")
+
+ if core.config == require("apisix.core.config_shdict") then
Review comment:
I think we may need to unify the term, the shdict is the container, the xds is the way, I think here `config_xds` will be better.
##########
File path: apisix/core/config_xds.lua
##########
@@ -0,0 +1,108 @@
+--
+-- Licensed to the Apache Software Foundation (ASF) under one or more
+-- contributor license agreements. See the NOTICE file distributed with
+-- this work for additional information regarding copyright ownership.
+-- The ASF licenses this file to You under the Apache License, Version 2.0
+-- (the "License"); you may not use this file except in compliance with
+-- the License. You may obtain a copy of the License at
+--
+-- http://www.apache.org/licenses/LICENSE-2.0
+--
+-- Unless required by applicable law or agreed to in writing, software
+-- distributed under the License is distributed on an "AS IS" BASIS,
+-- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+-- See the License for the specific language governing permissions and
+-- limitations under the License.
+--
+
+--- Get configuration form ngx.shared.DICT.
+--
+-- @module core.config_xds
+
+local base = require("resty.core.base")
+local config_local = require("apisix.core.config_local")
+local table = table
+local error = error
+local is_http = ngx.config.subsystem == "http"
+local string = string
+local io = io
+local package = package
+local new_tab = base.new_tab
+local ngx_timer_at = ngx.timer.at
+local ffi = require ("ffi")
+local C = ffi.C
+local router_config = ngx.shared["router-config"]
+
+local process
+if is_http then
+ process = require("ngx.process")
+end
+
+
+ffi.cdef[[
+extern void initial(void* router_zone_ptr);
+]]
+
+
+local _M = {
+ version = 0.1,
+ local_conf = config_local.local_conf,
+}
+
+
+-- todo: refactor this function in chash.lua and radixtree.lua
+local function load_shared_lib(lib_name)
+ local string_gmatch = string.gmatch
+ local string_match = string.match
+ local io_open = io.open
+ local io_close = io.close
Review comment:
Put them to the module level?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] spacewander commented on pull request #6614: feat: support reading configuration form xds(mvp)
Posted by GitBox <gi...@apache.org>.
spacewander commented on pull request #6614:
URL: https://github.com/apache/apisix/pull/6614#issuecomment-1071945183
IMHO, `external` is too general to know what it does. As etcd is an external service, what is the difference between `etcd` and `external`?
Moreover, we don't promise any API spec about the xds exchange format and we might optimize it for xds in the future. So I think using the name xds is specific enough for this feature.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org