[jira] [Created] (KARAF-4488) Memory leak in Karaf log LruList

["Amichai Rothman (JIRA)" <ji...@apache.org>]

Amichai Rothman created KARAF-4488:
--------------------------------------

             Summary: Memory leak in Karaf log LruList
                 Key: KARAF-4488
                 URL: https://issues.apache.org/jira/browse/KARAF-4488
             Project: Karaf
          Issue Type: Bug
          Components: karaf-shell
    Affects Versions: 4.0.4
            Reporter: Amichai Rothman
            Priority: Minor


The PaxAppender implementation in org.apache.karaf.log.core.internal.LruList has a memory management issue - although it is not an infinitely growing memory leak, it does hold on to arbitrarily large log strings in memory for an arbitrarily long time.

The current LruList implementation does limit the (configurable) number of log messages it keeps around in memory, but it does not limit their size or duration nor free them up when the system runs out of resources.

I came across this in a live system that got an OutOfMemoryError since it had some very large log lines (diagnostics/statistics dump) that were occasionally printed out to the log, but when there was little other log activity in between, ended up holding multiple instances of these very large log strings (MB) in memory simultaneously and eventually exhausting the JVM heap space. Although one might argue that an application shouldn't write such long log strings, this happens in practice, is allowed by the implementation, and people don't expect log strings to remain in memory for long and cause such trouble.

Btw, I'm not sure if the LruList is used for something other than the karaf shell command which shows recent log messages. Does it?

Some possible solutions:
- Limit the size of individual log messages (reject if too long).
- Limit the size of individual log messages (end with ellipsis after a given max length, or replace entire string with placeholder e.g. "<log message too long>").
- Limit the total memory used by all recent log messages, removing the older ones from the LRU list if it goes are above the limit - similar to how it currently removes messages when the limit on the number of messages to keep is reached.
- Use SoftReferences so that the memory is released if the system runs out of resources.
- Document this as being by design and blaming the client for writing large log messages.

In all options involving a maximum value, this can be made configurable like the max number of retained messages.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)