Improving FOP Performance?

[Ram Krishnamoorthi <ra...@yahoo.com>]

Hello,
We are using FOP to convert to generate PDF Invoices
for our Customers. We need to generate roughly around
25,000 Invoices Per Month and the Number would
increase each Month. We generate PDF Docs from XML
Input Files. We had Trial runs to check for
Performance and Memory usage.
Currently, We are just invoking FOP command directly
from a Shell Script which would call the FOP Command
repeatedly but making sure that not more than 2 (or
sometimes 3) FOP Processes are running at any instant.


I would like to have your opinion on the Performance
that we observe here. The Test Machine we run FOP on
has 2 CPUs and 4 GB RAM. Our Production Machine that
we would be running FOP on soon has 4 CPUs though

(1) In our case, An Invoice could have Several
Individual Sections. The Number of Sections could vary
from 10 to as high as 10,000 (depending on the data
present in the XML Input). On an Average, an Invoice
is around 10-20 Pages..while a small % of them exceed
500 Pages. Currently, We have a Page Sequence for 
each Section. We need to include only One Small Image
in each Invoice.

(2) There is a need to keep certain information
together in the same page, so I have used Tables
often..on an Average an Invoice could have around
50-60 Tables, But the Tables are not big (2 or 3 rows)
and have only one or 2 columns utmost. I do understand
Tables take up more memory..do they Impact Performance
significantly?

(3) I am using xsl:for-each in 3-4 places in the
Stylesheet to loop through nodes with same Tag instead
of using 'Template Match'..does that matter? 

At this Point, I see about 30,000 Pages being
generated in an Hour if I run 2 threads. I have not
embedded FOP in a Java Program Yet..Plan to do that..I
have seen 1 or 2 messages mentioning that Caching a
Stylesheet would help. Can someone share their
experience by doing that?

Are their other means to Improve Performance?   

Thanks for your assistance
Ram
 




		
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Helps protect you from nasty viruses. 
http://promotions.yahoo.com/new_mail

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org

Re: Improving FOP Performance?

["J.Pietschmann" <j3...@yahoo.de>]

Ram Krishnamoorthi wrote:
> Currently, We are just invoking FOP command directly
> from a Shell Script which would call the FOP Command

This means you have the overhead of starting a JVM process for
each individual invoice to process, including all the warm-up
costs. You should think about a custom Java program which
processes a larger batch of files, or about a server based
solution.

> I would like to have your opinion on the Performance
> that we observe here. The Test Machine we run FOP on
> has 2 CPUs and 4 GB RAM. Our Production Machine that
> we would be running FOP on soon has 4 CPUs though

There is an absolute limit on memory which a single JVM can manage.
Check the appropriate docs on how to set the memory limits for your
JVM. Optimize the settings in order to get most out of the available
physical RAM.
FOP uses a single thread for rendering a single PDF. You can only
take advantage of multiple CPUs by either running multiple JVMs in
parallel, or by using a custom Java program (command line or server)
which spawns multiple threads. For small input and output, and an
insignificant amount of images, FOP is processor bound rather than
I/O-bound. Including large, high resolution images or large user
font files (CJK) might turn FOP into an I/O-bound process. In any
case there shouldn't be much harm in starting a few more threads
than there are processors, unless you systematically run out of
JVM memory.

> Tables take up more memory..do they Impact Performance
> significantly?

Probably not. I'm not aware of a systematic study though.

> (3) I am using xsl:for-each in 3-4 places in the
> Stylesheet to loop through nodes with same Tag instead
> of using 'Template Match'..does that matter? 

FOP doesn't execute the transformation, you use an XSLT processor
for this purpose, usually the XSLT processor in the Java run time
library. Generally, for all XSLT processor commonly in use currently,
xsl:for-each is as fast as using xsl:apply-templates, and it may
be faster by a tiny, unnoticeable amount. Depending on the structure
f the input XML and the complexity of the transformation, using
xsl:apply-templates may be recommended for better maintainability.
BTW xsl:for-each is *not* a loop in the same sense as in common
procedural programming languages.

>  Caching a Stylesheet

Check the docs for javax.xml.transform.TransformerFactory.new Templates
and javax.xml.transform.TransformerFactory.Templates.new Transformer.
I doubt you'll notice much of an improvement unless your stylesheets
are somewhat complex and/or larger than a few hundred lines.

> Are their other means to Improve Performance?

It depends where the bottleneck is. In your current setup, eliminating
the JVM startup cost is most likely what will get you the most bang for
the buck. System utilities like vmstat and iostat, some simple
instrumentation, or a more advanced profiling will show you where
possible problems hide.

J.Pietschmann

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org

Re: Improving FOP Performance?

[je...@sanmargroup.com]

Hi Ram,

Please post the codes / send me the sample code files (.fo,.xsl)
for us to study..

Thanks
Eugene



Ram Krishnamoorthi <ra...@yahoo.com> 
11.02.2005 20:36
Please respond to
fop-user@xml.apache.org


To
fop-user@xml.apache.org
cc

Subject
Improving FOP Performance?






Hello,
We are using FOP to convert to generate PDF Invoices
for our Customers. We need to generate roughly around
25,000 Invoices Per Month and the Number would
increase each Month. We generate PDF Docs from XML
Input Files. We had Trial runs to check for
Performance and Memory usage.
Currently, We are just invoking FOP command directly
from a Shell Script which would call the FOP Command
repeatedly but making sure that not more than 2 (or
sometimes 3) FOP Processes are running at any instant.


I would like to have your opinion on the Performance
that we observe here. The Test Machine we run FOP on
has 2 CPUs and 4 GB RAM. Our Production Machine that
we would be running FOP on soon has 4 CPUs though

(1) In our case, An Invoice could have Several
Individual Sections. The Number of Sections could vary
from 10 to as high as 10,000 (depending on the data
present in the XML Input). On an Average, an Invoice
is around 10-20 Pages..while a small % of them exceed
500 Pages. Currently, We have a Page Sequence for 
each Section. We need to include only One Small Image
in each Invoice.

(2) There is a need to keep certain information
together in the same page, so I have used Tables
often..on an Average an Invoice could have around
50-60 Tables, But the Tables are not big (2 or 3 rows)
and have only one or 2 columns utmost. I do understand
Tables take up more memory..do they Impact Performance
significantly?

(3) I am using xsl:for-each in 3-4 places in the
Stylesheet to loop through nodes with same Tag instead
of using 'Template Match'..does that matter? 

At this Point, I see about 30,000 Pages being
generated in an Hour if I run 2 threads. I have not
embedded FOP in a Java Program Yet..Plan to do that..I
have seen 1 or 2 messages mentioning that Caching a
Stylesheet would help. Can someone share their
experience by doing that?

Are their other means to Improve Performance? 

Thanks for your assistance
Ram
 




 
__________________________________ 
Do you Yahoo!? 
Yahoo! Mail - Helps protect you from nasty viruses. 
http://promotions.yahoo.com/new_mail

---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org