You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apisix.apache.org by Qi Guo <gu...@gmail.com> on 2021/12/08 15:03:31 UTC
[Proposal] APISIX support file-log plugin
Hi Community,
I would like to add a File-log plugin to APISIX.
It appends the request and response data to the log file in JSON format.
The purpose is to better observe and analyze the status of each request
and response data.
The following are the functions of the plugin:
1. when the plugin is opened, each request is recorded in a separate
JSON object, split by a new \n.
2. you can specify your own output file path.
3. you can set whether to close and reopen the log file on each request,
if the file is not reopened, it should log all the requested data.
4. the value of the log field can be set or replaced by custom Lua code.
The File-log plugin has three configuration items:
{
……
type = "object",
properties = {
path = {type = “string”, required = true},
reopen = {
type = “boolean”,
required = true,
default = false
},
custom_fields_by_lua = {
type = "map"
}
}
}
Ref:
[1] https://docs.konghq.com/hub/kong-inc/file-log/
Do you have any ideas or suggestions for this? I would very much
look forward to your guidance and advice.
Best regards!
Re: [Proposal] APISIX support file-log plugin
Posted by Qi Guo <gu...@gmail.com>.
Hi, I need your help
I'm having some trouble with `C.close()`, which is declared as: `int
close(int fd)`.
I want to flush the cache of a file via the control API, by calling the
`C.close()`
method. I tried to use a global variable to store the return value of
`C.open()`,
but when I send a request to the control API, the `fd` is always empty.
What should I do to get the exact `fd` ?
Any better suggestions? Look forward to your guidance and advice!
Best regards!
Qi Guo <gu...@gmail.com> 于2021年12月13日周一 11:40写道：
> Get it,
> thank you for your advice, it has been very beneficial！
>
> Best regards!
>
> Zexuan Luo <sp...@apache.org> 于2021年12月10日周五 上午9:44写道：
>
>> The plugin can't be integrated with log-rotate as its fds are not
>> managed by Nginx.
>> We can develop a separate log-rotate feature in this plugin, or just
>> let users manage the lifecycle of the log file.
>>
>> Just drop some of my concerns here:
>> 1. maybe we can use file-logger as plugin name, as we already have
>> *-logger plugins
>> 2. like other loggers, we can use a custom log format via metadata.
>> See
>> https://github.com/apache/apisix/blob/c178435d7ada4eeb713d9a1688fb5f54f971abdf/apisix/plugins/http-logger.lua#L188
>> 3. reopen per request is too expensive, maybe we can register a
>> control API, and reopen part of the files when the API is hit.
>>
>> Qi Guo <gu...@gmail.com> 于2021年12月9日周四 16:22写道：
>> >
>> > Hi,
>> >
>> > > What’s the meaning for this? IMHO, it’ll add more overheads.
>> >
>> > My initial thought was that this would make it easier to develop and
>> debug
>> > locally
>> > to get a better view of the data being requested.
>> > This does add more overhead if there are too many requests. We cloud
>> open it
>> > only when we are debugging a problem.
>> >
>> > > Also, can we use it with the log-rotate plugin? Or how can we control
>> the
>> > file size?
>> >
>> > I'm sorry I missed this, can I pass the path of the file-log
>> configuration
>> > to log-rotate to keep it working.
>> > What do I need to do to do this and can you give me some good advice?
>> >
>> > Best wishes to you！
>> >
>> > Chao Zhang <zc...@gmail.com> 于2021年12月9日周四 上午9:51写道：
>> >
>> > > Hi,
>> > >
>> > > > you can set whether to close and reopen the log file on each request
>> > >
>> > > What’s the meaning for this? IMHO, it’ll add more overheads.
>> > >
>> > > Also, can we use it with the log-rotate plugin? Or how can we control
>> the
>> > > file size?
>> > >
>> > >
>> > > Chao Zhang
>> > > https://github.com/tokers
>> > >
>> > > On December 8, 2021 at 23:04:15, Qi Guo (guoqqqi@gmail.com) wrote:
>> > >
>> > > 3. you can set whether to close and reopen the log file on each
>> request,
>> > >
>> > >
>>
>
Re: [Proposal] APISIX support file-log plugin
Posted by Qi Guo <gu...@gmail.com>.
Get it,
thank you for your advice, it has been very beneficial！
Best regards!
Zexuan Luo <sp...@apache.org> 于2021年12月10日周五 上午9:44写道：
> The plugin can't be integrated with log-rotate as its fds are not
> managed by Nginx.
> We can develop a separate log-rotate feature in this plugin, or just
> let users manage the lifecycle of the log file.
>
> Just drop some of my concerns here:
> 1. maybe we can use file-logger as plugin name, as we already have
> *-logger plugins
> 2. like other loggers, we can use a custom log format via metadata.
> See
> https://github.com/apache/apisix/blob/c178435d7ada4eeb713d9a1688fb5f54f971abdf/apisix/plugins/http-logger.lua#L188
> 3. reopen per request is too expensive, maybe we can register a
> control API, and reopen part of the files when the API is hit.
>
> Qi Guo <gu...@gmail.com> 于2021年12月9日周四 16:22写道：
> >
> > Hi,
> >
> > > What’s the meaning for this? IMHO, it’ll add more overheads.
> >
> > My initial thought was that this would make it easier to develop and
> debug
> > locally
> > to get a better view of the data being requested.
> > This does add more overhead if there are too many requests. We cloud
> open it
> > only when we are debugging a problem.
> >
> > > Also, can we use it with the log-rotate plugin? Or how can we control
> the
> > file size?
> >
> > I'm sorry I missed this, can I pass the path of the file-log
> configuration
> > to log-rotate to keep it working.
> > What do I need to do to do this and can you give me some good advice?
> >
> > Best wishes to you！
> >
> > Chao Zhang <zc...@gmail.com> 于2021年12月9日周四 上午9:51写道：
> >
> > > Hi,
> > >
> > > > you can set whether to close and reopen the log file on each request
> > >
> > > What’s the meaning for this? IMHO, it’ll add more overheads.
> > >
> > > Also, can we use it with the log-rotate plugin? Or how can we control
> the
> > > file size?
> > >
> > >
> > > Chao Zhang
> > > https://github.com/tokers
> > >
> > > On December 8, 2021 at 23:04:15, Qi Guo (guoqqqi@gmail.com) wrote:
> > >
> > > 3. you can set whether to close and reopen the log file on each
> request,
> > >
> > >
>
Re: [Proposal] APISIX support file-log plugin
Posted by Qi Guo <gu...@gmail.com>.
Hi,
> What’s the meaning for this? IMHO, it’ll add more overheads.
My initial thought was that this would make it easier to develop and debug
locally
to get a better view of the data being requested.
This does add more overhead if there are too many requests. We cloud open it
only when we are debugging a problem.
> Also, can we use it with the log-rotate plugin? Or how can we control the
file size?
I'm sorry I missed this, can I pass the path of the file-log configuration
to log-rotate to keep it working.
What do I need to do to do this and can you give me some good advice?
Best wishes to you！
Chao Zhang <zc...@gmail.com> 于2021年12月9日周四 上午9:51写道：
> Hi,
>
> > you can set whether to close and reopen the log file on each request
>
> What’s the meaning for this? IMHO, it’ll add more overheads.
>
> Also, can we use it with the log-rotate plugin? Or how can we control the
> file size?
>
>
> Chao Zhang
> https://github.com/tokers
>
> On December 8, 2021 at 23:04:15, Qi Guo (guoqqqi@gmail.com) wrote:
>
> 3. you can set whether to close and reopen the log file on each request,
>
>
Re: [Proposal] APISIX support file-log plugin
Posted by Chao Zhang <zc...@gmail.com>.
Hi,
> you can set whether to close and reopen the log file on each request
What’s the meaning for this? IMHO, it’ll add more overheads.
Also, can we use it with the log-rotate plugin? Or how can we control the
file size?
Chao Zhang
https://github.com/tokers
On December 8, 2021 at 23:04:15, Qi Guo (guoqqqi@gmail.com) wrote:
3. you can set whether to close and reopen the log file on each request,