You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/07/29 15:35:15 UTC
[GitHub] [apisix] kys1230 opened a new issue, #7572: help request: 日志中输出自定义变量
kys1230 opened a new issue, #7572:
URL: https://github.com/apache/apisix/issues/7572
### Description
我需要在请求中通过自定义lua插件来生成一个统一变量。并希望将这个变量通过 file-logger 或 kafka-logger 等日志插件进行输出。
以前使用 openresty 是在 server {} 配置中通过 set $var_name ""; 定义了空变量后，在 rewite 阶段执行 lua 脚本以 ngx.var.$var_name = "xxx" 的方式实现变量的赋值，最终在日志中通过 $var_name 进行输出的。
但在 apisix 2.15.0 版本中我尝试使用同样的方法，得到的错误是
```
2022/07/29 15:26:00 [warn] 49#49: *26280 [lua] _G write guard:12: __newindex(): writing a global Lua variable ('$var_name') which may lead to race conditions between concurrent requests, so prefer the use of 'local' variables
stack traceback:
/usr/local/apisix/apisix/plugins/rongcloud-get-appkey.lua:50: in function 'phase_func'
/usr/local/apisix/apisix/plugin.lua:897: in function 'run_plugin'
/usr/local/apisix/apisix/plugin.lua:961: in function 'run_global_rules'
/usr/local/apisix/apisix/init.lua:425: in function 'http_access_phase'
access_by_lua(nginx.conf:287):2: in main chunk, client: 192.168.16.1, server: _, request: "HEAD / HTTP/1.1", host: "127.0.0.1:9080"
```
请问有什么其他方法可以更优雅的实现我的需求？感谢！
### Environment
- APISIX version (run `apisix version`):
2.15.0
- Operating system (run `uname -a`):
Linux c7341dd4c5e9 5.18.7-arch1-1 #1 SMP PREEMPT_DYNAMIC Sat, 25 Jun 2022 20:22:01 +0000 x86_64 Linux
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`):
nginx version: openresty/1.21.4.1
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`):
{"hostname":"c7341dd4c5e9","version":"2.15.0","boot_time":1659108142,"etcd_version":"3.4.0","id":"813de1fd-d3fe-45e4-84e4-462fce20cf9b"}
- APISIX Dashboard version, if relevant:
- Plugin runner version, for issues related to plugin runners:
自定义插件
- LuaRocks version, for installation issues (run `luarocks --version`):
/usr/local/bin/luarocks 3.8.0
LuaRocks main command-line interface
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: 日志中输出自定义变量
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1199701062
通过以下方式实现了。
I achieved it in the following ways.
```
local core = require("apisix.core")
local function getValue()
request "value"
end
core.ctx.register_var("var_name", function(ctx)
local var_a = getValue()
return var
end)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200426800
@tokers
The comments above describe my environment as minimal as possible.
---
上面的评论尽可能的最小化描述了我的环境。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200403424
@kys1230 Could you give us a minimal case for reproducing your problem?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200427519
Please don't ask me why I do this. The answer is service needs. I hope to generate variables at the beginning and use them directly by other plug-ins later, so as to improve processing efficiency, rather than each plug-in generates variables by itself.
---
请不要问我为什么这么做，答案是服务需要，我希望在最开始生成一次变量，后面的其他插件直接使用，从而提高处理效率，而不是每个插件自己生成变量。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] soulbird commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
soulbird commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200598633
Using `ctx.var.var_name`, what is result ? `nil` or `-` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 closed issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 closed issue #7572: help request: how to print custom variables in logs
URL: https://github.com/apache/apisix/issues/7572
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200426529
plugin-a.lua
```lua
local core = require("apisix.core")
local schema = {
type = "object",
}
local plugin_name = "plugin-a"
local _M = {
version = 0.1,
priority = 90,
name = plugin_name,
schema = schema,
}
function _M.check_schema(conf)
return core.schema.check(schema, conf)
end
local keyList = {"H1", "H2"}
local function get_value(headers)
-- The logic is simplified here, and the actual generated variables are more complex
-- 这里简化了逻辑，实际生成变量更复杂
for k, v in pairs(keyList) do
if headers[v] ~= nil then
return headers[v]
end
end
return nil
end
core.ctx.register_var("var_name", function(ctx)
local var_name = get_value(ngx.req.get_headers())
return var_name or "-"
end)
return _M
```
plugin-b.lua
```lua
local core = require("apisix.core")
local schema = {
type = "object",
}
local plugin_name = "plugin-b"
local _M = {
version = 0.1,
priority = 80,
name = plugin_name,
schema = schema,
}
function _M.check_schema(conf)
return core.schema.check(schema, conf)
end
function _M.rewrite(conf, ctx)
-- Here is just calling custom variables, which will actually have richer functions
-- 这里只是调用自定义变量，实际会有更丰富的功能
core.log.warn("plugin-a value: ", ctx.var_name)
end
return _M
```
apisix config.yaml
```yaml
plugins:
# The file-logger test is used here. In fact, I use kafka-logger
# 这里采用 file-logger 测试，实际上我使用的是 kafka-logger
- file-logger
- plugin-a
- plugin-b
```
set global rule
```bash
admin_url="http://127.0.0.1:9080"
admin_key="edd1c9f034335f136f87ad84b625c8f1"
curl -X PUT -H "X-API-KEY: $admin_key" $admin_url/apisix/admin/global_rules/1 -d '{
"plugins": {
"file-logger": {
"disable": false,
"path": "/usr/local/apisix/logs/file.log"
},
"plugin-a": {
disable": false
},
"plugin-b": {
"disable": false
}
}
}'
```
set file-logger logformat
```bash
admin_url="http://127.0.0.1:9080"
admin_key="edd1c9f034335f136f87ad84b625c8f1"
curl -H "X-API-KEY: $admin_key" $admin_url/apisix/admin/plugin_metadata/file-logger -X PUT -d '
{
"log_format": {
"host": "$host",
"@timestamp": "$time_iso8601",
"client_ip": "$remote_addr",
"app_key": "$app_key",
"status": "$status",
"request_time": "$request_time",
"upstream_addr": "$upstream_addr",
"upstream_response_time": "$upstream_response_time",
"var_name": "$var_name"
}
}'
```
view custom variables in the access log(file-logger)
```bash
$ curl -H 'H1: A' 127.0.0.1:9080
$ curl -H 'H2: B' 127.0.0.1:9080
$ curl -H 'H3: C' 127.0.0.1:9080
tail -3 /usr/local/apisix/logs/file.log | jq .var_name
"A"
"B"
"-"
```
view plugin-b log in error.log
```bash
$ grep /usr/local/apisix/logs/error.log | grep 'plugin-b.lua'
2022/07/31 13:24:47 [warn] 49#49: *27516 [lua] plugin-b.lua:21: phase_func(): plugin-a value: nil, client: 192.168.192.1, server: _, request: "GET / HTTP/1.1", host: "127.0.0.1:9080"
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200418838
当然。
---
of course.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tokers commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
tokers commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200559859
@tzssangglass @spacewander Could you take a look when you have time? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200589046
@tzssangglass That's what I do, but the question is how do I call this variable in other plug-ins?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 closed issue #7572: help request: 日志中输出自定义变量
Posted by GitBox <gi...@apache.org>.
kys1230 closed issue #7572: help request: 日志中输出自定义变量
URL: https://github.com/apache/apisix/issues/7572
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200600445
yes, the result is the same
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200582078
Among the several plugins you need to use this variable, choose the one with the highest priority to implement `register_var`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200624140
ctx.var.var_name can get it. It's strange that I'm checking the actual custom plug-in to see what's wrong.
---
可以通过 `ctx.var.var_name` 获取到我想要的内容了，很奇怪，我确定周末测试这种方法不好使，我在检查下实际使用的自定义插件看哪里出了问题。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] kys1230 commented on issue #7572: help request: how to print custom variables in logs
Posted by GitBox <gi...@apache.org>.
kys1230 commented on issue #7572:
URL: https://github.com/apache/apisix/issues/7572#issuecomment-1200227268
Sorry, I reopened this problem. Comments above can generate custom variables (var_name) in a custom plugin A and through `logformat $var_name ` print to log
But next, I want to read this custom variable (var_name) in the second custom plugin b to do additional functions. How can I get it?
Because reqeust body needs to be read to generate custom variables，so two plugins (plugin a priority 90, plugin b priority 80) run in the rewrite phase.
I try to use `ctx.var_name` or `ctx.var.var_name` in plugin b print custom variable (var_name), but they can't get the right results
---
抱歉，我重新打开了这个问题。上面的讨论已经可以在自定义插件 A 中生成自定义变量（var_name），并通过 logformat $var_name 打印到日志中。
但是接下来，我希望在自定义插件 B 中使用这个自定义变量 (var_name)，用于其他功能的判断依据，我应该如何访问它？
因为自定义变量的生成需要读取 request body ，所以我的两个自定义插件都运行在 rewitre 阶段，插件 A 的优先级是 90 , 插件 B 的优先级是 80
我尝试在插件 B 中通过 `ctx.var_name` 或 `ctx.var.var_name` 来打印这个自定义变量, 但是都没有得到正确的结果。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org