You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@trafficserver.apache.org by "Kit Chan (JIRA)" <ji...@apache.org> on 2016/04/15 10:05:25 UTC
[jira] [Work started] (TS-4266) ATS memory statistics shows that memory utilization is doubled after “traffic_ctlconfig reload”. And it is failed as it cannot find enough memory.
[ https://issues.apache.org/jira/browse/TS-4266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on TS-4266 started by Kit Chan.
------------------------------------
> ATS memory statistics shows that memory utilization is doubled after “traffic_ctlconfig reload”. And it is failed as it cannot find enough memory.
> --------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: TS-4266
> URL: https://issues.apache.org/jira/browse/TS-4266
> Project: Traffic Server
> Issue Type: Bug
> Components: Lua
> Reporter: Rajendra Kishore Bonumahanti
> Assignee: Kit Chan
> Fix For: sometime
>
>
> ATS memory statistics shows memory utilization is doubled after “traffic_ctl config reload”. We get “not enough memory” error in the subsequent attempt and “config reload” fails.
> ATS is configured with 100 map entries in remap.config, all share the same lua script.
> ATS is started: The memory information is..
> [root@mtanjv8cdnc73 trafficserver]# pmap -x 113330 | grep total
> total kB 1416092 670256 663736
> After 1st Config reload:
> [root@mtanjv8cdnc73 trafficserver]# pmap -x 113330 | grep total
> total kB 1932660 1128084 1121544
> After 2nd config reload: It had failed with error “not enough memory” and memory status as..
> [root@mtanjv8cdnc73 trafficserver]# pmap -x 113330 | grep total
> total kB 2170756 1167808 1160836
> Error displayed in diags.log:
> =======================
> [Mar 8 23:27:27.580] Server {0x2af92498b700} WARNING: Failed to create new instance for plugin /opt/trafficserver/libexec/trafficserver/tslua.so (not a TS_SUCCESS return)
> [Mar 8 23:27:27.580] Server {0x2af92498b700} WARNING: Could not add rule at line #3; Aborting!
> [Mar 8 23:27:27.580] Server {0x2af92498b700} WARNING: [ReverseProxy] Can't create new remap instance for plugin "/opt/trafficserver/libexec/trafficserver/tslua.so" - [ts_lua_add_module] luaL_loadfile /opt/trafficserver/etc/trafficserver/lua/process_remap.lua failed: not enough memory at line 3
> [Mar 8 23:27:27.580] Server {0x2af92498b700} WARNING: something failed during BuildTable() -- check your remap plugins!
> [Mar 8 23:27:27.595] Server {0x2af92498b700} WARNING: failed to reload remap.config, not replacing!
> Lua VM memory size at that time ,ts.debug(FUNCTION..'Lua VM memory: '..collectgarbage("count"))
> [Mar 8 23:27:27.579] Server {0x2af92498b700} DIAG: (ts_lua) __init__(): Lua VM memory: 3629.7060546875
> This shows that Lua VMs are hitting the max capacity.
> Solution:
> =======
> I looked at the ts_lua code TSRemapDeleteInstance () [ts_lua.c ] and ts_lua_del_module() [ts_lua_util.c] which does cleaning of the lua memory for the instance. However the lua memory is not released and reused.
> So, I have added code to start the garbage collector in ts_lua_del_module() .
> int
> ts_lua_del_module(ts_lua_instance_conf *conf, ts_lua_main_ctx *arr, int n)
> {
> ….
> lua_newtable(L);
> lua_replace(L, LUA_GLOBALSINDEX); /* L[GLOBAL] = EMPTY */
> lua_gc(L, LUA_GCCOLLECT, 0);
> TSMutexUnlock(arr[i].mutexp);
> }
> return 0;
> }
> This has improved the situation. However, I also added garbage collection in ts_lua_add_module() at the end. With these two additions, we have tested the code, the memory utilization is stable and we could do config reload at lest 100 times with the background load.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)