You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-dev@logging.apache.org by Gary Gregory <ga...@gmail.com> on 2016/03/14 21:41:33 UTC

StringBuilder limit 258 no good here

Hi all,

I've seem commits with a StringBuilder limit of 258 characters as opposed
to 1024 in the abstract string layout. Why the difference? Because the
former is in a ring buffer?

A casual check of my logs show line lengths of >259 as pretty common.

Gary

-- 
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

Re: StringBuilder limit 258 no good here

Posted by Matt Sicker <bo...@gmail.com>.
I didn't know the ring buffer was that big. The property-based method looks
good to me.

On 14 March 2016 at 16:40, Remko Popma <re...@gmail.com> wrote:

> This is now configurable (properties: log4j.initialReusableMsgSize and
> log4j.maxReusableMsgSize),
> default initial is 128 and the default max is now 518 (so max Ringbuffer
> memory consumption is 8MB by default).
> We could increase this, I'm open to suggestions.
>
> On Tue, Mar 15, 2016 at 8:24 AM, Remko Popma <re...@gmail.com>
> wrote:
>
>> I agree this should be configurable.
>>
>> My thinking was to avoid consuming too much memory with the RingBuffer:
>> if an occasional huge message causes the StringBuilder to grow this memory
>> would be held on forever by the RingBuffer. Not sure what the sweet spot is
>> here.
>>
>> Some numbers:
>> In GC-free mode, there are 8000 slots in the ring buffer and each slot
>> has a StringBuilder to hold a reusable message.
>> We initially size these StringBuilders to be 128 characters, so 256 bytes.
>> 256 * 8000 = 2 MB of memory consumed by the StringBuilders in the
>> RingBuffer (4MB if they all resize).
>>
>>
>>
>>
>> On Tue, Mar 15, 2016 at 7:49 AM, Gary Gregory <ga...@gmail.com>
>> wrote:
>>
>>> I think we just need a reasonable default and it must be configurable.
>>>
>>> Here, I have a lot of cases where objects are toString()'ed in DEBUG and
>>> TRACE mode and that usually blows away a 258 limit.
>>>
>>> Gary
>>>
>>> On Mon, Mar 14, 2016 at 1:44 PM, Matt Sicker <bo...@gmail.com> wrote:
>>>
>>>> OpenJDK uses 8192 byte buffers internally a lot last time I checked.
>>>> Might be overkill, though 258 does sound pretty low.
>>>>
>>>> On 14 March 2016 at 15:41, Gary Gregory <ga...@gmail.com> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I've seem commits with a StringBuilder limit of 258 characters as
>>>>> opposed to 1024 in the abstract string layout. Why the difference? Because
>>>>> the former is in a ring buffer?
>>>>>
>>>>> A casual check of my logs show line lengths of >259 as pretty common.
>>>>>
>>>>> Gary
>>>>>
>>>>> --
>>>>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
>>>>> Java Persistence with Hibernate, Second Edition
>>>>> <http://www.manning.com/bauer3/>
>>>>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>>>>> Spring Batch in Action <http://www.manning.com/templier/>
>>>>> Blog: http://garygregory.wordpress.com
>>>>> Home: http://garygregory.com/
>>>>> Tweet! http://twitter.com/GaryGregory
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Matt Sicker <bo...@gmail.com>
>>>>
>>>
>>>
>>>
>>> --
>>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
>>> Java Persistence with Hibernate, Second Edition
>>> <http://www.manning.com/bauer3/>
>>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>>> Spring Batch in Action <http://www.manning.com/templier/>
>>> Blog: http://garygregory.wordpress.com
>>> Home: http://garygregory.com/
>>> Tweet! http://twitter.com/GaryGregory
>>>
>>
>>
>


-- 
Matt Sicker <bo...@gmail.com>

Re: StringBuilder limit 258 no good here

Posted by Remko Popma <re...@gmail.com>.
This is now configurable (properties: log4j.initialReusableMsgSize and
log4j.maxReusableMsgSize),
default initial is 128 and the default max is now 518 (so max Ringbuffer
memory consumption is 8MB by default).
We could increase this, I'm open to suggestions.

On Tue, Mar 15, 2016 at 8:24 AM, Remko Popma <re...@gmail.com> wrote:

> I agree this should be configurable.
>
> My thinking was to avoid consuming too much memory with the RingBuffer: if
> an occasional huge message causes the StringBuilder to grow this memory
> would be held on forever by the RingBuffer. Not sure what the sweet spot is
> here.
>
> Some numbers:
> In GC-free mode, there are 8000 slots in the ring buffer and each slot has
> a StringBuilder to hold a reusable message.
> We initially size these StringBuilders to be 128 characters, so 256 bytes.
> 256 * 8000 = 2 MB of memory consumed by the StringBuilders in the
> RingBuffer (4MB if they all resize).
>
>
>
>
> On Tue, Mar 15, 2016 at 7:49 AM, Gary Gregory <ga...@gmail.com>
> wrote:
>
>> I think we just need a reasonable default and it must be configurable.
>>
>> Here, I have a lot of cases where objects are toString()'ed in DEBUG and
>> TRACE mode and that usually blows away a 258 limit.
>>
>> Gary
>>
>> On Mon, Mar 14, 2016 at 1:44 PM, Matt Sicker <bo...@gmail.com> wrote:
>>
>>> OpenJDK uses 8192 byte buffers internally a lot last time I checked.
>>> Might be overkill, though 258 does sound pretty low.
>>>
>>> On 14 March 2016 at 15:41, Gary Gregory <ga...@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I've seem commits with a StringBuilder limit of 258 characters as
>>>> opposed to 1024 in the abstract string layout. Why the difference? Because
>>>> the former is in a ring buffer?
>>>>
>>>> A casual check of my logs show line lengths of >259 as pretty common.
>>>>
>>>> Gary
>>>>
>>>> --
>>>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
>>>> Java Persistence with Hibernate, Second Edition
>>>> <http://www.manning.com/bauer3/>
>>>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>>>> Spring Batch in Action <http://www.manning.com/templier/>
>>>> Blog: http://garygregory.wordpress.com
>>>> Home: http://garygregory.com/
>>>> Tweet! http://twitter.com/GaryGregory
>>>>
>>>
>>>
>>>
>>> --
>>> Matt Sicker <bo...@gmail.com>
>>>
>>
>>
>>
>> --
>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
>> Java Persistence with Hibernate, Second Edition
>> <http://www.manning.com/bauer3/>
>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>> Spring Batch in Action <http://www.manning.com/templier/>
>> Blog: http://garygregory.wordpress.com
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory
>>
>
>

Re: StringBuilder limit 258 no good here

Posted by Remko Popma <re...@gmail.com>.
I agree this should be configurable.

My thinking was to avoid consuming too much memory with the RingBuffer: if
an occasional huge message causes the StringBuilder to grow this memory
would be held on forever by the RingBuffer. Not sure what the sweet spot is
here.

Some numbers:
In GC-free mode, there are 8000 slots in the ring buffer and each slot has
a StringBuilder to hold a reusable message.
We initially size these StringBuilders to be 128 characters, so 256 bytes.
256 * 8000 = 2 MB of memory consumed by the StringBuilders in the
RingBuffer (4MB if they all resize).




On Tue, Mar 15, 2016 at 7:49 AM, Gary Gregory <ga...@gmail.com>
wrote:

> I think we just need a reasonable default and it must be configurable.
>
> Here, I have a lot of cases where objects are toString()'ed in DEBUG and
> TRACE mode and that usually blows away a 258 limit.
>
> Gary
>
> On Mon, Mar 14, 2016 at 1:44 PM, Matt Sicker <bo...@gmail.com> wrote:
>
>> OpenJDK uses 8192 byte buffers internally a lot last time I checked.
>> Might be overkill, though 258 does sound pretty low.
>>
>> On 14 March 2016 at 15:41, Gary Gregory <ga...@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I've seem commits with a StringBuilder limit of 258 characters as
>>> opposed to 1024 in the abstract string layout. Why the difference? Because
>>> the former is in a ring buffer?
>>>
>>> A casual check of my logs show line lengths of >259 as pretty common.
>>>
>>> Gary
>>>
>>> --
>>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
>>> Java Persistence with Hibernate, Second Edition
>>> <http://www.manning.com/bauer3/>
>>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>>> Spring Batch in Action <http://www.manning.com/templier/>
>>> Blog: http://garygregory.wordpress.com
>>> Home: http://garygregory.com/
>>> Tweet! http://twitter.com/GaryGregory
>>>
>>
>>
>>
>> --
>> Matt Sicker <bo...@gmail.com>
>>
>
>
>
> --
> E-Mail: garydgregory@gmail.com | ggregory@apache.org
> Java Persistence with Hibernate, Second Edition
> <http://www.manning.com/bauer3/>
> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
> Spring Batch in Action <http://www.manning.com/templier/>
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>

Re: StringBuilder limit 258 no good here

Posted by Gary Gregory <ga...@gmail.com>.
I think we just need a reasonable default and it must be configurable.

Here, I have a lot of cases where objects are toString()'ed in DEBUG and
TRACE mode and that usually blows away a 258 limit.

Gary

On Mon, Mar 14, 2016 at 1:44 PM, Matt Sicker <bo...@gmail.com> wrote:

> OpenJDK uses 8192 byte buffers internally a lot last time I checked. Might
> be overkill, though 258 does sound pretty low.
>
> On 14 March 2016 at 15:41, Gary Gregory <ga...@gmail.com> wrote:
>
>> Hi all,
>>
>> I've seem commits with a StringBuilder limit of 258 characters as opposed
>> to 1024 in the abstract string layout. Why the difference? Because the
>> former is in a ring buffer?
>>
>> A casual check of my logs show line lengths of >259 as pretty common.
>>
>> Gary
>>
>> --
>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
>> Java Persistence with Hibernate, Second Edition
>> <http://www.manning.com/bauer3/>
>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>> Spring Batch in Action <http://www.manning.com/templier/>
>> Blog: http://garygregory.wordpress.com
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory
>>
>
>
>
> --
> Matt Sicker <bo...@gmail.com>
>



-- 
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

Re: StringBuilder limit 258 no good here

Posted by Matt Sicker <bo...@gmail.com>.
OpenJDK uses 8192 byte buffers internally a lot last time I checked. Might
be overkill, though 258 does sound pretty low.

On 14 March 2016 at 15:41, Gary Gregory <ga...@gmail.com> wrote:

> Hi all,
>
> I've seem commits with a StringBuilder limit of 258 characters as opposed
> to 1024 in the abstract string layout. Why the difference? Because the
> former is in a ring buffer?
>
> A casual check of my logs show line lengths of >259 as pretty common.
>
> Gary
>
> --
> E-Mail: garydgregory@gmail.com | ggregory@apache.org
> Java Persistence with Hibernate, Second Edition
> <http://www.manning.com/bauer3/>
> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
> Spring Batch in Action <http://www.manning.com/templier/>
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>



-- 
Matt Sicker <bo...@gmail.com>