Document insertion causing memory stress

[Alan Deikman <Al...@znyx.com>]

In an earlier thread I documented how my application merges one 
SVGElement into a main document.   It works, but when I do it multiple 
times my application slows down and crashes, most often with an "out of 
heap space" exception.  The SVG documents are not that complex (edited 
with inkscape).  The inserted document has about 6800 Nodes and no text 
elements.

In trying to figure out what is going on, I ran the Netbeans profiler to 
discover what objects are being created and by whom.  What I found was 
an astounding number of DOMMutationEvent objects (over 64,000) and also 
a huge number of HashMap$KeyIterator objects (over 27,000 using 890KB of 
memory), mostly created by 
SVGTextElementBridge$AttributedStringBuffer.toAttributedString();

I realize this is probably not enough detail to diagnose anything, but 
does this suggest anything to anyone about what I should be looking for 
or what I might be doing wrong?

-- 
Alan Deikman



---------------------------------------------------------------------
To unsubscribe, e-mail: batik-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-users-help@xmlgraphics.apache.org

Re: Document insertion causing memory stress

[Alan Deikman <Al...@znyx.com>]

Helder Magalhães wrote:
> Well, for the heap space issue, you should really consider increasing
> the memory limit allocated to the JVM (the default is not even enough
> for medium complexity graphics, "medium" being naturally a very
> subjective measurement). ;-)  Lots of threads on this subject on the
> mailing list (and even a few in the bug tracker) -- guess a FAQ [1]
> item for this would made sense, even if this is not related with
> Batik, as it's a common Java misconception...
>   
Well I did end up increasing the memory available to the JVM -- probably 
later than I should have.   That's where the "subjective" you mentioned 
comes in -- I wasn't prepared to accept the amount of memory my 
application would take.  The fully assembled graphic to me is not really 
complex (one parent graphic, and about 16 grafted-in graphics) but the 
Batik-related objects (as far as I can tell from the Netbeans profiler) 
are taking up about 120MB to render it -- and that's after GC.

If I were doing this all with Swing natively it would be much harder to 
program to get the same feature set but it would take about 1/10th the 
memory.  At least the result with Batik looks great and performs well.  
That's a win.

At least I know now that my program logic is correct -- and that I am 
not up against some sort of Batik "bug" that I am ill-prepared to figure 
out.  

> The good news is that all of this should be expected. :-)  As you said
> to be merging a document into another one, then the main document's
> DOM will be updated: that explains the high number of
> "DOMMutationEvent" objects. As we're at it, you may consider taking a
> look at this rather interesting and powerful mechanism [2] [3]. ;-)
> I'd say that the "HashMap$KeyIterator" objects correspond to pointers
> to elements and element attributes of the document being merged. The
> high number of both of these objects are likely coherent with the
> almost 7000 nodes you said the document being merged contained
> (probably is the sum of element attributes and maybe of elements
> also).
>   
This does seem to be the case.  However I suspect that many of the 
HashMaps floating around also have something to do with the GC process 
-- a byproduct of instantiating and throwing away a lot of objects.  I 
just don't know enough about it to be sure.

> The less good news is that I'm not sure how to workaround this. I'm
> fairly convinced that you can (temporarily) disable the DOM Mutation
> events from being triggered (crawling through the examples and
> documentation might help) or, if you don't need them, maybe completely
> disable this functionality (note that I'm also not sure if this would
> break Batik internally!). Disabling the DOM Mutation events should
> lead to a temporary performance improvement and memory decrease while
> doing the merge. As for the "HashMap$KeyIterator", I don't think you
> can do much about it, but that's also far beyond my knowledge of
> Batik... ;-)
>   
This solution did occur to me -- what would be called for is an 
optimized importNode method that avoids this problem.  What might also 
work better is to just build an SVGDocument from scratch each time and 
invoking a Builder (I don't think DOM mutations occur in that process).

 From this experience what I would ask the Batik developers for is some 
way of monitoring and controlling memory usage.  It is easy enough to 
get an object count given a Node parameter, but what is really needed is 
a way to get the concrete implementation to fess up on what objects it 
has instantiated and what memory they use.

When I have some time I am going to look deeper into this.   For now 
since I am up and running (memory is cheap, right?) I have to set this 
aside and focus on my business logic. 

> Hope this helps,
>  Helder
>   
You always seem to have a useful link or two.   Thanks for the pointers.

> [1] http://xmlgraphics.apache.org/batik/faq.html
> [2] http://www.xml.com/pub/a/2002/10/09/mutate.html
> [3] http://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-eventgroupings-mutationevents
>
>   
-- 
Alan Deikman



---------------------------------------------------------------------
To unsubscribe, e-mail: batik-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-users-help@xmlgraphics.apache.org

Re: Document insertion causing memory stress

[Helder Magalhães <he...@gmail.com>]

Hi Alan,


> In an earlier thread I documented how my application merges one SVGElement
> into a main document.   It works, but when I do it multiple times my
> application slows down and crashes, most often with an "out of heap space"
> exception.  The SVG documents are not that complex (edited with inkscape).
>  The inserted document has about 6800 Nodes and no text elements.

Well, for the heap space issue, you should really consider increasing
the memory limit allocated to the JVM (the default is not even enough
for medium complexity graphics, "medium" being naturally a very
subjective measurement). ;-)  Lots of threads on this subject on the
mailing list (and even a few in the bug tracker) -- guess a FAQ [1]
item for this would made sense, even if this is not related with
Batik, as it's a common Java misconception...


> In trying to figure out what is going on, I ran the Netbeans profiler to
> discover what objects are being created and by whom.  What I found was an
> astounding number of DOMMutationEvent objects (over 64,000) and also a huge
> number of HashMap$KeyIterator objects (over 27,000 using 890KB of memory),
> mostly created by
> SVGTextElementBridge$AttributedStringBuffer.toAttributedString();

Well, from a quick look it seems like there are good news and less
good (~=bad?) news. :-D

The good news is that all of this should be expected. :-)  As you said
to be merging a document into another one, then the main document's
DOM will be updated: that explains the high number of
"DOMMutationEvent" objects. As we're at it, you may consider taking a
look at this rather interesting and powerful mechanism [2] [3]. ;-)
I'd say that the "HashMap$KeyIterator" objects correspond to pointers
to elements and element attributes of the document being merged. The
high number of both of these objects are likely coherent with the
almost 7000 nodes you said the document being merged contained
(probably is the sum of element attributes and maybe of elements
also).

The less good news is that I'm not sure how to workaround this. I'm
fairly convinced that you can (temporarily) disable the DOM Mutation
events from being triggered (crawling through the examples and
documentation might help) or, if you don't need them, maybe completely
disable this functionality (note that I'm also not sure if this would
break Batik internally!). Disabling the DOM Mutation events should
lead to a temporary performance improvement and memory decrease while
doing the merge. As for the "HashMap$KeyIterator", I don't think you
can do much about it, but that's also far beyond my knowledge of
Batik... ;-)


> I realize this is probably not enough detail to diagnose anything, but does
> this suggest anything to anyone about what I should be looking for or what I
> might be doing wrong?

Yeah, not much information to work with... :-|  Note that the source
of all this can be in the approach you have chosen to perform the
document merge itself -- there may be more than one way to accomplish
it...


Hope this helps,
 Helder


[1] http://xmlgraphics.apache.org/batik/faq.html
[2] http://www.xml.com/pub/a/2002/10/09/mutate.html
[3] http://www.w3.org/TR/DOM-Level-2-Events/events.html#Events-eventgroupings-mutationevents

---------------------------------------------------------------------
To unsubscribe, e-mail: batik-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: batik-users-help@xmlgraphics.apache.org