Memory leak in delimiter scanning

[Steve Lawrence <sl...@apache.org>]

Daffodil is currently used in a long running process (via a NiFi
processor). After running it for a while, it looks like there is a
memory leak in delimiter scanning, but since I'm not too familiar with
this particular code I wanted to run it by dev before making a fix.

We have a pool of Registers that contain state information for delimiter
scanning (e.g. result string, matched delimiter string, characters
read). These fields are usually reset after being retrieved from the
pool by a call to reset().

One of these fields is "charsReadUntilDelim", but it is currently not
reset like other StringBuilders in the Register.  It is allocated once
when a Register is created and is only ever appended to. Since it is
never reset, it essentially grows forever one character at a time. It
does not appear to ever be used aside from append()s.

Does anyone know the purpose of this charsReadUntilDelim? It does not
appear to be used anymore. Perhaps it was used for debugging at some
point but is no longer necessary? The easy fix for the memory leak it to
just reset() it, but if it's not used anywhere we might as well just
remove it.

Re: Memory leak in delimiter scanning

[Taylor Wise <tw...@gmail.com>]

Looks like it is safe to remove.

On Wed, Nov 29, 2017 at 11:37 AM, Steve Lawrence <sl...@apache.org>
wrote:

> Daffodil is currently used in a long running process (via a NiFi
> processor). After running it for a while, it looks like there is a
> memory leak in delimiter scanning, but since I'm not too familiar with
> this particular code I wanted to run it by dev before making a fix.
>
> We have a pool of Registers that contain state information for delimiter
> scanning (e.g. result string, matched delimiter string, characters
> read). These fields are usually reset after being retrieved from the
> pool by a call to reset().
>
> One of these fields is "charsReadUntilDelim", but it is currently not
> reset like other StringBuilders in the Register.  It is allocated once
> when a Register is created and is only ever appended to. Since it is
> never reset, it essentially grows forever one character at a time. It
> does not appear to ever be used aside from append()s.
>
> Does anyone know the purpose of this charsReadUntilDelim? It does not
> appear to be used anymore. Perhaps it was used for debugging at some
> point but is no longer necessary? The easy fix for the memory leak it to
> just reset() it, but if it's not used anywhere we might as well just
> remove it.
>



-- 
-Taylor Wise