[jira] Issue Comment Edited: (VELOCITY-223) VMs that use a large number of directives and macros use excessive amounts of memory - over 4-6MB RAM per form

["Will Glass-Husain (JIRA)" <de...@velocity.apache.org>]

    [ https://issues.apache.org/jira/browse/VELOCITY-223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12613199#action_12613199 ] 

wglass edited comment on VELOCITY-223 at 7/13/08 1:34 PM:
---------------------------------------------------------------------

Hi,

Thanks for all contributions on this old but important issue.

I've dug into this a bit.  Alexey's succinct analysis is correct (also described by Lei).  Fundamentally, the issue is that every rendering of a template with macros creates a new String object containing the body of the macro.

Alexey asks "Why is it implemented this way"?  From a common-sense perpective, it seems like it would make more sense to parse the macro body, then share a common parse tree among all templates which use the macro.

There are two reasons this is difficult
(1) Macros are dynamically included at runtime, not parse-time.  Different macros (with the same name) may be included with #parse

(2) Templates are cached.  I haven't investigated this in too much depth, but it seems likely to me that if the code shares a parse tree among different templates (or different templates included with #parse) it may run into caching/update issues.

This is a long-winded way of noting that I think the simplest solution is Lei's patch.  Originally I thought this was a litle kludgy, but now I like the way it drastically saves memory without imposing the effort of redoing the existing macro mechanism.   I'm setting up some "before" and "after" tests to verify the performance claims, but I just wanted to note I think this is the right way to go.


      was (Author: wglass):
    Hi,

Thanks for all contributions on this old but important issue.

I've dug into this a bit.  Alexey's succinct analysis is correct (also described by Lei).  Fundamentally, the issue is that every rendering of a template with macros creates a new String object containing the body of the macro.

Alexey asks "Why is it implemented this way"?  From a common-sense perpective, it seems like it would make more sense to parse the macro body, then share a common parse tree among all templates which use the macro.

There are two reasons this is difficult
(1) Macros are dynamically included at runtime, not parse-time.  Different macros (with the same name) may be included with #parse

(2) Templates are cached.  I haven't investigated this in too much depth, but it seems likely to me that if the code shares a parse tree among different templates (or different templates included with #parse) it may run into caching/update issues.

This is a long-winded way of note that I think the simplest solution is Lei's patch.  Originally I thought this was a litle kludgy, but now I like the way it drastically saves memory without imposing the effort of redoing the existing macro mechanism.   I'm setting up some "before" and "after" tests to verify the performance claims, but I just wanted to note I think this is the right way to go.

  
> VMs that use a large number of directives and macros use excessive amounts of memory - over 4-6MB RAM per form
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: VELOCITY-223
>                 URL: https://issues.apache.org/jira/browse/VELOCITY-223
>             Project: Velocity
>          Issue Type: Bug
>          Components: Engine
>    Affects Versions: 1.3.1
>         Environment: Operating System: All
> Platform: All
>            Reporter: Christian Nichols
>             Fix For: 1.6
>
>         Attachments: 223-patch.txt, AllVelocityMemoryByClass.html, StringImagePool.java, VelocityCharStream.java, VelocityMemory.JPG
>
>
> Our application FinanceCenter is based on Velocity as the template engine.  We 
> have a library of about 200 macros and about 400 VM files.  Because the 
> velocity parser copies the macro body into the VM during parsing, macros that 
> are frequently used (even though identical and using local contexts) use up 
> large amounts of memory.  On our Linux server (running Redhat 7.2 with Sun JDK 
> 1.4.1_04) we can easily use up over 1GB of RAM simply by opening up many forms 
> (about 150) - the server starts out using 60MB after startup.  This memory 
> times out after 5 minutes and is returned which tells me that it is screen 
> memory.  Our problem is that the NT JVM and Linux JVM (32 bit) are currently 
> limited to about 1.6 - 2.0 GB of ram for heap space.  Thus, using a fair number 
> of forms in the application leaves little space for user session data.
> We have implemented a caching mechanism for compiled templates and integrated 
> it into Velocity so that cached objects are timed out of the cache but the 
> server is still using large amounts of memory.  We finally had to rewrite many 
> of our macros into Java so that memory usage would be reduced (note that these 
> macros were doing complex screen formatting not business logic).  Doing this 
> has reduced our memory by about 30%.  This is currently our biggest issue with 
> Velocity and is causing us to review our decision to stay with Velocity going 
> forward.  This is because we will likely end up with close to 1,000 forms by 
> the end of next year and need to know that Velocity can deal with this.  Is 
> there any work underway to share compiled macro AST's?  This would greatly 
> reduce the amount of memory used.  I have reviewed the parser code that is 
> doing this but it seems that this is an embedded part of the design and not 
> easily changed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@velocity.apache.org
For additional commands, e-mail: dev-help@velocity.apache.org