You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-dev@logging.apache.org by luca boncompagni <lb...@gmail.com> on 2016/03/11 17:38:53 UTC
issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Hi to all,
I'm using lo4j2-2.5 and I have some memory usage problems with
AbstractStringLayout.getStringBuilder(). If I write a very long line
of log, the memory used by this line of log is never cleaned.
In the code, I see result.setLength(0) that sets the current size to
0, but the buffer of the string builder remains as long as it is.
Should be ok to use something like this:
result.setLength(0);
result.setLength(DEFAULT_STRING_BUILDER_SIZE);
result.trimToSize();
I have another question, why we should use a threadlocal to get an
empty StringBuilder?
Regards,
Luca
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by Remko Popma <re...@gmail.com>.
It totally makes sense to trim it. With the upcoming GC-free changes I was
planning to reduce the RingBuffer's StringBuilder to 518 chars (two resizes
from the default). There are a lot of those instances so it makes sense to
be strict about their size. With the Layout there are only as many
StringBuilders are there are threads, so we could keep their size a bit
larger. Maybe ~2K but make it configurable?
On Sat, Mar 12, 2016 at 8:49 AM, Gary Gregory <ga...@gmail.com>
wrote:
> So the default trim would be 10K, seems reasonable.
>
> Gary
>
> On Fri, Mar 11, 2016 at 1:02 PM, Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>> I would trim it to a default max value - something like 10K. Even that
>> seems like a lot. This should be configurable through our properties file.
>>
>> Ralph
>>
>> On Mar 11, 2016, at 1:56 PM, Gary Gregory <ga...@gmail.com> wrote:
>>
>> So do we need a layout setting to shrink the buffer back down after a
>> while to a user defined level?
>>
>> This is after answering the obvious question of why would you log a 10 MB
>> message?
>>
>> Gary
>>
>> On Fri, Mar 11, 2016 at 12:38 PM, luca boncompagni <lb...@gmail.com>
>> wrote:
>>
>>>
>>> Il 11/mar/2016 21:14, "Matt Sicker" <bo...@gmail.com> ha scritto:
>>> >
>>> > Does your app make a lot of large log messages or is it just used in a
>>> couple loggers? Because I think you could use a custom MessageFactory for
>>> those loggers to reduce memory usage at the cost of performance.
>>> >
>>>
>>> My app is a webapp running on wildfly. Long message are not so frequent,
>>> but if I have every thread of wildfly with a StringBuilder of ~ 10Mb I have
>>> some serious memory problem.
>>>
>>> Luca
>>>
>>
>>
>>
>> --
>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
>> Java Persistence with Hibernate, Second Edition
>> <http://www.manning.com/bauer3/>
>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>> Spring Batch in Action <http://www.manning.com/templier/>
>> Blog: http://garygregory.wordpress.com
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory
>>
>>
>>
>
>
> --
> E-Mail: garydgregory@gmail.com | ggregory@apache.org
> Java Persistence with Hibernate, Second Edition
> <http://www.manning.com/bauer3/>
> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
> Spring Batch in Action <http://www.manning.com/templier/>
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by Gary Gregory <ga...@gmail.com>.
So the default trim would be 10K, seems reasonable.
Gary
On Fri, Mar 11, 2016 at 1:02 PM, Ralph Goers <ra...@dslextreme.com>
wrote:
> I would trim it to a default max value - something like 10K. Even that
> seems like a lot. This should be configurable through our properties file.
>
> Ralph
>
> On Mar 11, 2016, at 1:56 PM, Gary Gregory <ga...@gmail.com> wrote:
>
> So do we need a layout setting to shrink the buffer back down after a
> while to a user defined level?
>
> This is after answering the obvious question of why would you log a 10 MB
> message?
>
> Gary
>
> On Fri, Mar 11, 2016 at 12:38 PM, luca boncompagni <lb...@gmail.com>
> wrote:
>
>>
>> Il 11/mar/2016 21:14, "Matt Sicker" <bo...@gmail.com> ha scritto:
>> >
>> > Does your app make a lot of large log messages or is it just used in a
>> couple loggers? Because I think you could use a custom MessageFactory for
>> those loggers to reduce memory usage at the cost of performance.
>> >
>>
>> My app is a webapp running on wildfly. Long message are not so frequent,
>> but if I have every thread of wildfly with a StringBuilder of ~ 10Mb I have
>> some serious memory problem.
>>
>> Luca
>>
>
>
>
> --
> E-Mail: garydgregory@gmail.com | ggregory@apache.org
> Java Persistence with Hibernate, Second Edition
> <http://www.manning.com/bauer3/>
> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
> Spring Batch in Action <http://www.manning.com/templier/>
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
>
>
--
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by luca boncompagni <lb...@gmail.com>.
2016-03-11 22:02 GMT+01:00 Ralph Goers <ra...@dslextreme.com>:
> I would trim it to a default max value - something like 10K. Even that seems
> like a lot. This should be configurable through our properties file.
>
> Ralph
>
Can we use something like:
protected static final int DEFAULT_STRING_BUILDER_RESIZE = 10 *
DEFAULT_STRING_BUILDER_SIZE;
protected static StringBuilder getStringBuilder() {
StringBuilder result = threadLocal.get();
if (result == null
|| result.length() > DEFAULT_STRING_BUILDER_RESIZE) {
result = new StringBuilder(DEFAULT_STRING_BUILDER_SIZE);
threadLocal.set(result);
}
result.setLength(0);
return result;
}
Luca
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by Ralph Goers <ra...@dslextreme.com>.
I would trim it to a default max value - something like 10K. Even that seems like a lot. This should be configurable through our properties file.
Ralph
> On Mar 11, 2016, at 1:56 PM, Gary Gregory <ga...@gmail.com> wrote:
>
> So do we need a layout setting to shrink the buffer back down after a while to a user defined level?
>
> This is after answering the obvious question of why would you log a 10 MB message?
>
> Gary
>
> On Fri, Mar 11, 2016 at 12:38 PM, luca boncompagni <lbonco@gmail.com <ma...@gmail.com>> wrote:
>
> Il 11/mar/2016 21:14, "Matt Sicker" <boards@gmail.com <ma...@gmail.com>> ha scritto:
> >
> > Does your app make a lot of large log messages or is it just used in a couple loggers? Because I think you could use a custom MessageFactory for those loggers to reduce memory usage at the cost of performance.
> >
>
> My app is a webapp running on wildfly. Long message are not so frequent, but if I have every thread of wildfly with a StringBuilder of ~ 10Mb I have some serious memory problem.
>
> Luca
>
>
>
> --
> E-Mail: garydgregory@gmail.com <ma...@gmail.com> | ggregory@apache.org <ma...@apache.org>
> Java Persistence with Hibernate, Second Edition <http://www.manning.com/bauer3/>
> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
> Spring Batch in Action <http://www.manning.com/templier/>
> Blog: http://garygregory.wordpress.com <http://garygregory.wordpress.com/>
> Home: http://garygregory.com/ <http://garygregory.com/>
> Tweet! http://twitter.com/GaryGregory <http://twitter.com/GaryGregory>
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by luca boncompagni <lb...@gmail.com>.
2016-03-11 21:56 GMT+01:00 Gary Gregory <ga...@gmail.com>:
> So do we need a layout setting to shrink the buffer back down after a while
> to a user defined level?
>
> This is after answering the obvious question of why would you log a 10 MB
> message?
>
> Gary
>
Hi, in my web-app I have to call some third part service, I need to
log the response.
They use soap web service and my question is why they should ever
design a soap service with a so large response.
Luca
---------------------------------------------------------------------
To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
For additional commands, e-mail: log4j-dev-help@logging.apache.org
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by Gary Gregory <ga...@gmail.com>.
So do we need a layout setting to shrink the buffer back down after a while
to a user defined level?
This is after answering the obvious question of why would you log a 10 MB
message?
Gary
On Fri, Mar 11, 2016 at 12:38 PM, luca boncompagni <lb...@gmail.com> wrote:
>
> Il 11/mar/2016 21:14, "Matt Sicker" <bo...@gmail.com> ha scritto:
> >
> > Does your app make a lot of large log messages or is it just used in a
> couple loggers? Because I think you could use a custom MessageFactory for
> those loggers to reduce memory usage at the cost of performance.
> >
>
> My app is a webapp running on wildfly. Long message are not so frequent,
> but if I have every thread of wildfly with a StringBuilder of ~ 10Mb I have
> some serious memory problem.
>
> Luca
>
--
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by luca boncompagni <lb...@gmail.com>.
Il 11/mar/2016 21:14, "Matt Sicker" <bo...@gmail.com> ha scritto:
>
> Does your app make a lot of large log messages or is it just used in a
couple loggers? Because I think you could use a custom MessageFactory for
those loggers to reduce memory usage at the cost of performance.
>
My app is a webapp running on wildfly. Long message are not so frequent,
but if I have every thread of wildfly with a StringBuilder of ~ 10Mb I have
some serious memory problem.
Luca
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by Matt Sicker <bo...@gmail.com>.
Does your app make a lot of large log messages or is it just used in a
couple loggers? Because I think you could use a custom MessageFactory for
those loggers to reduce memory usage at the cost of performance.
On 11 March 2016 at 14:08, luca boncompagni <lb...@gmail.com> wrote:
>
> Il 11/mar/2016 20:17, "Gary Gregory" <ga...@gmail.com> ha scritto:
> >
> > On Fri, Mar 11, 2016 at 8:38 AM, luca boncompagni <lb...@gmail.com>
> wrote:
> >>
> >> Hi to all,
> >>
> >> I'm using lo4j2-2.5 and I have some memory usage problems with
> >> AbstractStringLayout.getStringBuilder(). If I write a very long line
> >> of log, the memory used by this line of log is never cleaned.
> >>
> >> In the code, I see result.setLength(0) that sets the current size to
> >> 0, but the buffer of the string builder remains as long as it is.
> >>
> >> Should be ok to use something like this:
> >>
> >> result.setLength(0);
> >> result.setLength(DEFAULT_STRING_BUILDER_SIZE);
> >> result.trimToSize();
> >>
> >> I have another question, why we should use a threadlocal to get an
> >> empty StringBuilder?
> >
> >
> > Performance. It saves a lot of time and memory to avoid creating a new
> StringBuilder for each log event.
> >
> > Gary
>
> Ok, I'll do some test.
>
> Can you help me about memory usage?
>
> Luca
>
--
Matt Sicker <bo...@gmail.com>
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by luca boncompagni <lb...@gmail.com>.
Il 11/mar/2016 20:17, "Gary Gregory" <ga...@gmail.com> ha scritto:
>
> On Fri, Mar 11, 2016 at 8:38 AM, luca boncompagni <lb...@gmail.com>
wrote:
>>
>> Hi to all,
>>
>> I'm using lo4j2-2.5 and I have some memory usage problems with
>> AbstractStringLayout.getStringBuilder(). If I write a very long line
>> of log, the memory used by this line of log is never cleaned.
>>
>> In the code, I see result.setLength(0) that sets the current size to
>> 0, but the buffer of the string builder remains as long as it is.
>>
>> Should be ok to use something like this:
>>
>> result.setLength(0);
>> result.setLength(DEFAULT_STRING_BUILDER_SIZE);
>> result.trimToSize();
>>
>> I have another question, why we should use a threadlocal to get an
>> empty StringBuilder?
>
>
> Performance. It saves a lot of time and memory to avoid creating a new
StringBuilder for each log event.
>
> Gary
Ok, I'll do some test.
Can you help me about memory usage?
Luca
Re: issue with org.apache.logging.log4j.core.layout.AbstractStringLayout.getStringBuilder()
Posted by Gary Gregory <ga...@gmail.com>.
On Fri, Mar 11, 2016 at 8:38 AM, luca boncompagni <lb...@gmail.com> wrote:
> Hi to all,
>
> I'm using lo4j2-2.5 and I have some memory usage problems with
> AbstractStringLayout.getStringBuilder(). If I write a very long line
> of log, the memory used by this line of log is never cleaned.
>
> In the code, I see result.setLength(0) that sets the current size to
> 0, but the buffer of the string builder remains as long as it is.
>
> Should be ok to use something like this:
>
> result.setLength(0);
> result.setLength(DEFAULT_STRING_BUILDER_SIZE);
> result.trimToSize();
>
> I have another question, why we should use a threadlocal to get an
> empty StringBuilder?
>
Performance. It saves a lot of time and memory to avoid creating a new
StringBuilder for each log event.
Gary
>
> Regards,
> Luca
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: log4j-dev-unsubscribe@logging.apache.org
> For additional commands, e-mail: log4j-dev-help@logging.apache.org
>
>
--
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory