You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fop-users@xmlgraphics.apache.org by "Puppala, Kumar (LNG-CON)" <ku...@lexisnexis.com> on 2008/03/05 21:09:04 UTC
RE: Fop 0.20.5 vs Fop Trunk Performace
Hi Andreas,
I ran our server using DevPartner and noticed a lot of memory leaks.
After much analysis, I was able to determine that the manner in which we
embed FOP in our application (for that matter any application) might be
resulting in these leaks. The specific piece of code that I found very
interesting is as shown below:
// Setup JAXP using identity transformer
TransformerFactory factory =
TransformerFactory.newInstance();
Transformer transformer = factory.newTransformer(); //
identity transformer
When you use the TransformerFactory in this manner (within each run),
the SAXParser instance created by the transformer is held in memory. On
subsequent runs, I noticed more and more instances of SAXParser's in
memory. To fix this, I moved this code to the calling function, thus
keeping a single copy of transformer. This greatly reduced the GC times.
However, I am still seeing an incremental increase rendering times on
subsequent runs. I am trying to streamline the code on our end to
adhere to your suggestions. If you find anything, please let us know.
Thanks,
Kumar
-----Original Message-----
From: Andreas Delmelle [mailto:andreas.delmelle@telenet.be]
Sent: Wednesday, February 20, 2008 12:47 PM
To: fop-users@xmlgraphics.apache.org
Subject: Re: Fop 0.20.5 vs Fop Trunk Performace
On Feb 15, 2008, at 20:37, Puppala, Kumar (LNG-CON) wrote:
Hi Kumar
<snip />
> Well, it's not so much the number of objects I'm thinking of, but
> rather, how much time is spent executing specific methods and which
> ones take longer in later iterations. The actual cause of the
> slowdown may precisely be located in a class of which there are
> relatively few instances alive, if I judge correctly. Or did you
> already check whether the bulk of the increase in processing-time is
> really only spent on garbage-collection?
>
> --> I did verify that garbage collection is playing a very
> significant role in these increased timings. I am currently using
> the Java Profiling tool to identify the methods where we have the
> maximum execution time. I ran the basic simple.fo file provided in
> the examples directory and below is the screen shot from the
> profiling tool:
>
> <image001.jpg>
>
> I need to do further analysis on this but comparing this output to
> the one generated by running the same fo file against fop 0.20.5
> codebase, I do see two significant areas (lineBreak and layoutMgr)
> that are taking time. The profiling output for the same test on fop
> 0.20.5 codebase is as shown below:
>
Unfortunately, a single run is far from representative, especially
when compared to the set of tests you ran earlier.
What one could say, based on the screenshot, is that most of the time
is spent on parsing the FO and on creating loggers, but then we would
be conveniently ignoring the simple facts that:
* all loggers are static, so the cost of creating them is overly
represented if you measure only a single run (hence why LogFactory is
at the top of the list here)
* the org.apache.fop.fo package contains a lot of static
initialization as well, so ... (unfortunately, no details shown in
the screenshot to confirm)
In the meantime, I have run some local tests using the files you
published. Thanks for those!
The tests were run as an iteration of the most basic example
ExampleFO2PDF.java, slightly modified to process the whole set of
files you provided one-by-one, using a specified maximum number of
concurrently running threads.
So far, I have run:
1 batch of 1500 runs, single-threaded
1 batch of 1500 runs, in two threads
1 batch of 1500 runs, in three threads
1 batch of 3000 runs, in 10 threads.
I did not immediately see any indication of individual tests
consistently taking longer in later iterations. Each of the 117 files
was processed about 13 times in the course of one batch of 1500, 26
times for the batch of 3000. Some run a tiny bit slower, but IMO not
enough to count as actually 'slower'. Some tests are even faster the
13th time around than the first time.
I do see some runs taking significantly longer, most likely due to
garbage-collection. This effect can already be observed when running
in a single thread.
As potential improvements:
There are some tests that generate a significant amount of log-
messages (mostly warnings). AFAICT, if you can make your generated FO
such that it produces less of these warnings, there could already be
a noticeable speed-benefit. The processing times in my tests were
always a lot higher for those tests that generated a lot of warnings.
To reduce the strain on the garbage-collector, you could also
consider generating 'better' FO.
Of course, this is a matter of opinion, but one striking example I
encountered is the pattern:
<fo:block space-after="1em"/>
<fo:block text-indent="0.250000in">
<fo:inline ...font-properties....>#PCDATA</fo:inline>
</fo:block>
<fo:block space-after="1em"/>
The above construct is (in most situations) equivalent to:
<fo:block space-before="1em" space-after="1em"
text-indent="0.250000in"
...font-properties...>#PCDATA</fo:block>
IOW, by inserting empty fo:blocks and extra fo:inlines, the first
pattern, which is extensively used in the test-FOs, generates a lot
of Java objects which are, strictly speaking, completely unnecessary.
If you could iron that out, you're also bound to see an improvement.
As rules of thumb:
- empty fo:blocks should be avoided at any cost; sometimes, preserved
linefeeds generate the same effect with much less overhead
- fo:inlines should only be used when absolutely necessary (to change
certain properties inline); in the above example, it serves no
particular purpose apart from carrying the font-related properties,
while those could just as well be specified on the surrounding fo:block.
I'll keep running some more tests and will report back if I have any
news.
Cheers
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
RE: Fop 0.20.5 vs Fop Trunk Performace
Posted by "Puppala, Kumar (LNG-CON)" <ku...@lexisnexis.com>.
I forgot to mention this point. We in fact are running in multi-threaded
mode and I am creating a pool of TranformerFactories and transformers.
Each thread needs to go through a synchronized call to get access to
them. As you pointed out, TransformerFactory is not thread-safe and I
found it the hard way (got a lot of NullPointerExceptions when I was not
using the pool).
Thanks !!
-----Original Message-----
From: Andreas Delmelle [mailto:andreas.delmelle@telenet.be]
Sent: Wednesday, March 05, 2008 5:07 PM
To: fop-users@xmlgraphics.apache.org
Subject: Re: Fop 0.20.5 vs Fop Trunk Performace
On Mar 5, 2008, at 21:36, Andreas Delmelle wrote:
> On Mar 5, 2008, at 21:09, Puppala, Kumar (LNG-CON) wrote:
>>
>> // Setup JAXP using identity transformer
>> TransformerFactory factory =
>> TransformerFactory.newInstance();
>> Transformer transformer = factory.newTransformer(); //
>> identity transformer
>
> Interesting! So the memory leaks are not inherent to FOP it would
> seem, but to the way the JAXP-pattern is used.
>
>>
>> When you use the TransformerFactory in this manner (within each
>> run), the SAXParser instance created by the transformer is held in
>> memory. On subsequent runs, I noticed more and more instances of
>> SAXParser's in memory. To fix this, I moved this code to the
>> calling function, thus keeping a single copy of transformer.
>
> Makes sense. The TransformerFactory should indeed (better practice)
> be declared only once. If you ever plan on using XSLT stylesheets,
> you could also cache a Templates object.
This made me re-read the API docs, and it this may suit your
particular setup, I was wrong and in general one should actually be
*very* careful when re-using either the TransformerFactory or the
Transformer. Neither of the objects are thread-safe, so this can only
be done if you are absolutely certain that no two threads will
concurrently access either of them... Re-using them in sequence is no
problem whatsoever, and actually seems to be recommended, based on
your observations.
Cheers
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Fop 0.20.5 vs Fop Trunk Performace
Posted by Andreas Delmelle <an...@telenet.be>.
On Mar 5, 2008, at 21:36, Andreas Delmelle wrote:
> On Mar 5, 2008, at 21:09, Puppala, Kumar (LNG-CON) wrote:
>>
>> // Setup JAXP using identity transformer
>> TransformerFactory factory =
>> TransformerFactory.newInstance();
>> Transformer transformer = factory.newTransformer(); //
>> identity transformer
>
> Interesting! So the memory leaks are not inherent to FOP it would
> seem, but to the way the JAXP-pattern is used.
>
>>
>> When you use the TransformerFactory in this manner (within each
>> run), the SAXParser instance created by the transformer is held in
>> memory. On subsequent runs, I noticed more and more instances of
>> SAXParser’s in memory. To fix this, I moved this code to the
>> calling function, thus keeping a single copy of transformer.
>
> Makes sense. The TransformerFactory should indeed (better practice)
> be declared only once. If you ever plan on using XSLT stylesheets,
> you could also cache a Templates object.
This made me re-read the API docs, and it this may suit your
particular setup, I was wrong and in general one should actually be
*very* careful when re-using either the TransformerFactory or the
Transformer. Neither of the objects are thread-safe, so this can only
be done if you are absolutely certain that no two threads will
concurrently access either of them... Re-using them in sequence is no
problem whatsoever, and actually seems to be recommended, based on
your observations.
Cheers
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org
Re: Fop 0.20.5 vs Fop Trunk Performace
Posted by Andreas Delmelle <an...@telenet.be>.
On Mar 5, 2008, at 21:09, Puppala, Kumar (LNG-CON) wrote:
Hi Kumar
> I ran our server using DevPartner and noticed a lot of memory
> leaks. After much analysis, I was able to determine that the manner
> in which we embed FOP in our application (for that matter any
> application) might be resulting in these leaks. The specific piece
> of code that I found very interesting is as shown below:
>
> // Setup JAXP using identity transformer
> TransformerFactory factory =
> TransformerFactory.newInstance();
> Transformer transformer = factory.newTransformer(); //
> identity transformer
Interesting! So the memory leaks are not inherent to FOP it would
seem, but to the way the JAXP-pattern is used.
>
> When you use the TransformerFactory in this manner (within each
> run), the SAXParser instance created by the transformer is held in
> memory. On subsequent runs, I noticed more and more instances of
> SAXParser’s in memory. To fix this, I moved this code to the
> calling function, thus keeping a single copy of transformer.
Makes sense. The TransformerFactory should indeed (better practice)
be declared only once. If you ever plan on using XSLT stylesheets,
you could also cache a Templates object.
> This greatly reduced the GC times. However, I am still seeing an
> incremental increase rendering times on subsequent runs. I am
> trying to streamline the code on our end to adhere to your
> suggestions. If you find anything, please let us know.
First try focusing on reducing the log-messages. Another post of
today demonstrated that this helps a lot in increasing the overall
processing time.
I currently have only little time to run the tests, but if I do, I'll
certainly report back.
Thanks for the feedback!
Cheers
Andreas
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-users-unsubscribe@xmlgraphics.apache.org
For additional commands, e-mail: fop-users-help@xmlgraphics.apache.org