You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/12/06 06:23:55 UTC
[GitHub] [apisix] ucg-shin opened a new issue, #8461: help request: using ext-plugins on requiredBody true, memory leak occurs
ucg-shin opened a new issue, #8461:
URL: https://github.com/apache/apisix/issues/8461
### Description
When using the ext-plugins with the requiredBody true option, the openresty memory continues to increase(more than 500MB, there are 8 worker processes, more than total 4 GB are used).
When using the ext-plugins with the requiredBody false option, it maintains around 30-40 megabytes.
(around 2000 TPS, content size is varies )
The memory increase was resolved by modifying the handle_extra_info function in the apisix/plugins/ext-plugin/init.lua file.
* original source
` if packed_res then
extra_info_resp.AddResult(builder, packed_res)
end
local resp = extra_info_resp.End(builder)
builder:Finish(resp)
return builder:Output()
`
* modified source
` if packed_res then
extra_info_resp.AddResult(builder, packed_res)
end
local resp = extra_info_resp.End(builder)
builder:Finish(resp)
local out = builder:Output()
builder = nil -- release
builder = flatbuffers.Builder(0) -- reallocation
return out
`
it works normally and maintains 30~40 megabytes of openresty memory.
Probably the problem is that builder:Clear() doesn't release memory.
The Release or Reset function of flatbuffer builder (https://google.github.io/flatbuffers/classflatbuffers_1_1_flat_buffer_builder.html#a5e2c3a1e47f0ab4a3a3f07a07a52fb45) is not in the builder.lua file.
Any better solution? and Could source modifications be a problem?
### Environment
- APISIX version (run `apisix version`): 2.99.0
- Operating system (run `uname -a`): rhel 8
- OpenResty / Nginx version (run `openresty -V` or `nginx -V`): 1.21.4
- etcd version, if relevant (run `curl http://127.0.0.1:9090/v1/server_info`):3.5.4
- APISIX Dashboard version, if relevant: 2.13.0
- Plugin runner version, for issues related to plugin runners: java plugin runner 0.3.0
- LuaRocks version, for installation issues (run `luarocks --version`): 5.4.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [apisix] tzssangglass commented on issue #8461: help request: using ext-plugins on requiredBody true, memory leak occurs
Posted by GitBox <gi...@apache.org>.
tzssangglass commented on issue #8461:
URL: https://github.com/apache/apisix/issues/8461#issuecomment-1340332465
ref: https://github.com/google/flatbuffers/blob/master/lua/flatbuffers/builder.lua#L64-L79
```lua
-- Clears the builder and resets the state. It does not actually clear the backing binary array, it just reuses it as
-- needed. This is a performant way to use the builder for multiple constructions without the overhead of multiple
-- builder allocations.
function mt:Clear()
self.finished = false
self.nested = false
self.minalign = 1
self.currentVTable = nil
self.objectEnd = nil
self.head = self.bytes.size -- place the head at the end of the binary array
-- clear vtables instead of making a new table
local vtable = self.vtables
local vtableCount = #vtable
for i=1,vtableCount do vtable[i] = nil end
end
```
what you say `doesn't release memory.` means `It does not actually clear the backing binary array`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] help request: using ext-plugins on requiredBody true, memory leak occurs [apisix]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #8461:
URL: https://github.com/apache/apisix/issues/8461#issuecomment-1845045440
This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] help request: using ext-plugins on requiredBody true, memory leak occurs [apisix]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #8461:
URL: https://github.com/apache/apisix/issues/8461#issuecomment-1822460762
This issue has been marked as stale due to 350 days of inactivity. It will be closed in 2 weeks if no further activity occurs. If this issue is still relevant, please simply write any comment. Even if closed, you can still revive the issue at any time or discuss it on the dev@apisix.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] help request: using ext-plugins on requiredBody true, memory leak occurs [apisix]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #8461: help request: using ext-plugins on requiredBody true, memory leak occurs
URL: https://github.com/apache/apisix/issues/8461
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org