You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fop-users@xmlgraphics.apache.org by Ram Krishnamoorthi <ra...@yahoo.com> on 2005/02/28 19:45:18 UTC
Upper Limit on XML File Size?
Hello,
Apologize for posting this in the FOP Forum..but does
anyone know if there any Upper Limit on the Size of
the XML File that can be Processed by the Xalan XSLT
Processor?
I am having trouble generating simple FO Objects for
XML Files which are over 50 MB. I dont get any error
messages.but when i observe the FO output file..I can
see that it has not Transformed the entire XML File.
Any help here is greatly appreciated.
Thanks
Ram
__________________________________
Do you Yahoo!?
Yahoo! Mail - Easier than ever with enhanced search. Learn more.
http://info.mail.yahoo.com/mail_250
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org
Re: Upper Limit on XML File Size?
Posted by Ram Krishnamoorthi <ra...@yahoo.com>.
I am calling the Transformation from the command line
and attempting to create a FO Object
xalan.sh -IN <XMLFile> -XSL <XSLFile> -OUT
<FOFileName>
I am using JDK 1.4
The Xalan Version is an older one.
java rg.apache.xalan.processor.XSLProcessorVersion
Xalan Java 2.2.D11
Thanks Jan ..I am now looking at the MulberryTech XSLT
Forum as well.
--- "J.Pietschmann" <j3...@yahoo.de> wrote:
> Ram Krishnamoorthi wrote:
> > My Largest XML File is about 65 MB. I am
> allocating a
> > Heap Size of 2 GB and still not able to Transform
> the
> > Entire File. I can see that the Java Process
> > performing the Transformation uses about 1.7 GB
> during
> > the Transformation..and my Stylesheet does not
> have
> > any Fancy Transformation, all it does is apply the
> > same Template repeatedly..No Tables or Images
> included
> > as well.
>
> I'm confused. XSLT is not about tables or images.
>
>
> > Increasing the Max Heap Size beyond 2 GB didnt
> help
> > either.
>
> The Sun JVM has a hard upper limit, which used to be
> 1G for heap and another one for the stack.
>
> If the XSLT processor really runs out of memory, you
> should get a notice somewhere. Maybe somewhere where
> you didn't look yet.
> Some questions:
> - What JVM are you using? Which version?
> - What Xalan release are you using? How do you know
> you
> are really using this release?
> - How do you run the XSLT processor? Command line?
> Embedded?
> - Are you sure you really mean you are running only
> the
> transformation?
> - How do you handle the output? Ordinary serializing
> to a
> file?
>
> You might also investigate in using a memory
> profiler (I
> like DrMem - small, free, easy to use), or asking on
> the
> Xalan user list.
>
> J.Pietschmann
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> fop-user-unsubscribe@xml.apache.org
> For additional commands, e-mail:
> fop-user-help@xml.apache.org
>
>
__________________________________
Do you Yahoo!?
Yahoo! Mail - 250MB free storage. Do more. Manage less.
http://info.mail.yahoo.com/mail_250
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org
Re: Upper Limit on XML File Size?
Posted by "J.Pietschmann" <j3...@yahoo.de>.
Ram Krishnamoorthi wrote:
> My Largest XML File is about 65 MB. I am allocating a
> Heap Size of 2 GB and still not able to Transform the
> Entire File. I can see that the Java Process
> performing the Transformation uses about 1.7 GB during
> the Transformation..and my Stylesheet does not have
> any Fancy Transformation, all it does is apply the
> same Template repeatedly..No Tables or Images included
> as well.
I'm confused. XSLT is not about tables or images.
> Increasing the Max Heap Size beyond 2 GB didnt help
> either.
The Sun JVM has a hard upper limit, which used to be
1G for heap and another one for the stack.
If the XSLT processor really runs out of memory, you
should get a notice somewhere. Maybe somewhere where
you didn't look yet.
Some questions:
- What JVM are you using? Which version?
- What Xalan release are you using? How do you know you
are really using this release?
- How do you run the XSLT processor? Command line? Embedded?
- Are you sure you really mean you are running only the
transformation?
- How do you handle the output? Ordinary serializing to a
file?
You might also investigate in using a memory profiler (I
like DrMem - small, free, easy to use), or asking on the
Xalan user list.
J.Pietschmann
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org
Re: Upper Limit on XML File Size?
Posted by Ram Krishnamoorthi <ra...@yahoo.com>.
Thanks Jan for the Response
My Largest XML File is about 65 MB. I am allocating a
Heap Size of 2 GB and still not able to Transform the
Entire File. I can see that the Java Process
performing the Transformation uses about 1.7 GB during
the Transformation..and my Stylesheet does not have
any Fancy Transformation, all it does is apply the
same Template repeatedly..No Tables or Images included
as well.
Increasing the Max Heap Size beyond 2 GB didnt help
either.
Is there someother Memory setting that should be
hiked?
Thanks again
--- "J.Pietschmann" <j3...@yahoo.de> wrote:
> Ram Krishnamoorthi wrote:
> > Apologize for posting this in the FOP Forum..but
> does
> > anyone know if there any Upper Limit on the Size
> of
> > the XML File that can be Processed by the Xalan
> XSLT
> > Processor?
> >
> > I am having trouble generating simple FO Objects
> for
> > XML Files which are over 50 MB. I dont get any
> error
> > messages.but when i observe the FO output file..I
> can
> > see that it has not Transformed the entire XML
> File.
>
> The maximal input file size is mostly determined by
> the available
> memory, and to some extent by the complexity off the
> transformation
> and output handling.
> Xalan (as well as most other XSLT processors, most
> of the time)
> reads the complete input into memory and builds an
> internal data
> structure for processing. As a rule of thumb, JVM
> memory to input
> files size is 10:1 (unless recent releases improved
> significantly),
> i.e. a 50MB input will need around 500MB JVM heap
> size for holding
> the input. The transformation also uses memory,
> especially building
> RTF variables, recursive templates for complex
> string manipulation,
> large key tables, XPath expressions which select a
> significant
> part of the input but not a whole subtree and
> Muenchean grouping
> can eat a lot of additional memory. If you serialize
> the output,
> this is usually streamed, but if you want to process
> the result
> further, you'll need even more ressources..
>
> J.Pietschmann
>
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> fop-user-unsubscribe@xml.apache.org
> For additional commands, e-mail:
> fop-user-help@xml.apache.org
>
>
__________________________________
Do you Yahoo!?
Yahoo! Mail - Easier than ever with enhanced search. Learn more.
http://info.mail.yahoo.com/mail_250
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org
Re: Upper Limit on XML File Size?
Posted by "J.Pietschmann" <j3...@yahoo.de>.
Ram Krishnamoorthi wrote:
> Apologize for posting this in the FOP Forum..but does
> anyone know if there any Upper Limit on the Size of
> the XML File that can be Processed by the Xalan XSLT
> Processor?
>
> I am having trouble generating simple FO Objects for
> XML Files which are over 50 MB. I dont get any error
> messages.but when i observe the FO output file..I can
> see that it has not Transformed the entire XML File.
The maximal input file size is mostly determined by the available
memory, and to some extent by the complexity off the transformation
and output handling.
Xalan (as well as most other XSLT processors, most of the time)
reads the complete input into memory and builds an internal data
structure for processing. As a rule of thumb, JVM memory to input
files size is 10:1 (unless recent releases improved significantly),
i.e. a 50MB input will need around 500MB JVM heap size for holding
the input. The transformation also uses memory, especially building
RTF variables, recursive templates for complex string manipulation,
large key tables, XPath expressions which select a significant
part of the input but not a whole subtree and Muenchean grouping
can eat a lot of additional memory. If you serialize the output,
this is usually streamed, but if you want to process the result
further, you'll need even more ressources..
J.Pietschmann
---------------------------------------------------------------------
To unsubscribe, e-mail: fop-user-unsubscribe@xml.apache.org
For additional commands, e-mail: fop-user-help@xml.apache.org