You are viewing a plain text version of this content. The canonical link for it is here.
Posted to fop-dev@xmlgraphics.apache.org by Edward Dowgiallo <ed...@ptdprolog.net> on 2001/12/06 19:00:45 UTC

Memory utilization

I am attempting to do database publishing using fop to format the output. I have created a style sheet that creates a table of contents and an index for my document using <fo:basic-link internal-destination=...> in combination with id attributes. A larger document is represented by an XML file of approximately 8MB and contains thousands of such links. I am finding that the JVM memory pool required to process this is about 750MB at this point. My database is not yet complete and the documents are likely to grow both in size and number of links a good bit before I am done.

I have tried running this on a machine with 512MB of memory and it trashed hopelessly for 12 hours without completing. I have 1 machine with 1GB of RAM that I normally reserve as a server that can do the processing in 1/2 an hour, but I am worried that I will lose the ability to create the document as it continues to grow.

Do you have any suggestions that would help tame the memory utilization? Will the next release of fop be able to process links with a smaller amount of memory?

Thank you,
Ed

RE: Memory utilization

Posted by Matthias Fischer <m....@abc-media.de>.
Hi Ed,

I enclosed a mail by Lloyd McKenzie (lmckenzi@ca.ibm.com) into this one.
Unfortunately, he didn't send any XSL code with it.
At abc.Mediaservice, we aren't through with implementing the solution yet.
However, we started on the input side to provide our headings H1..H[n] with
unique id's.

Maybe, Lloyd, you could consider to send Ed and me a piece of XSL code to
process the data, it would probably spare us some programming time.

Matthias
  -----Original Message-----
  From: Edward Dowgiallo [mailto:edowgial@ptdprolog.net]
  Sent: Friday, December 07, 2001 3:43 PM
  To: fop-dev@xml.apache.org
  Subject: Re: Memory utilization


  Matthias,

      I am willing to try this. My documents do contain a lot of section
headers. Can you provide me with more detail? Perhaps a syntax example of a
header of the "certain level."

  Ed
    ----- Original Message -----
    From: Matthias Fischer
    To: fop-dev@xml.apache.org
    Sent: Friday, December 07, 2001 3:00 AM
    Subject: RE: Memory utilization


    Hi Ed,

    we ran into a similar problem a couple of weeks ago. The most suitable
solution that was offered then by our colleagues in the list was to portion
the transformation process. FOP seems to hold its breath until it dives out
of the waters of transformation at the other end of its page master
sequence.
    In our case, it wasn't mass data but an editorial document with lots of
graphics, in reference to which I was advised to use headers (of a certain
level) to start a new page master sequence.

    I think, some way or the other you could find a similar solution.

    Matthias


    Dott. Matthias Fischer
    abc.Mediaservice GmbH

    Nebelhornstra�e 8
    86807 Buchloe
    Tel. (08241) 9686-38
    Fax  (08241) 9686-26
    http://www.abc-media.de
    e-mail: m.fischer@abc-media.de

Re: Memory utilization

Posted by Edward Dowgiallo <ed...@ptdprolog.net>.
Matthias,

    I am willing to try this. My documents do contain a lot of section headers. Can you provide me with more detail? Perhaps a syntax example of a header of the "certain level."

Ed
  ----- Original Message ----- 
  From: Matthias Fischer 
  To: fop-dev@xml.apache.org 
  Sent: Friday, December 07, 2001 3:00 AM
  Subject: RE: Memory utilization


  Hi Ed,

  we ran into a similar problem a couple of weeks ago. The most suitable solution that was offered then by our colleagues in the list was to portion the transformation process. FOP seems to hold its breath until it dives out of the waters of transformation at the other end of its page master sequence.
  In our case, it wasn't mass data but an editorial document with lots of graphics, in reference to which I was advised to use headers (of a certain level) to start a new page master sequence.

  I think, some way or the other you could find a similar solution.

  Matthias


  Dott. Matthias Fischer
  abc.Mediaservice GmbH

  Nebelhornstraße 8
  86807 Buchloe
  Tel. (08241) 9686-38
  Fax  (08241) 9686-26
  http://www.abc-media.de
  e-mail: m.fischer@abc-media.de


    -----Original Message-----
    From: Edward Dowgiallo [mailto:edowgial@ptdprolog.net]
    Sent: Thursday, December 06, 2001 7:01 PM
    To: fop-dev@xml.apache.org
    Subject: Memory utilization


    I am attempting to do database publishing using fop to format the output. I have created a style sheet that creates a table of contents and an index for my document using <fo:basic-link internal-destination=...> in combination with id attributes. A larger document is represented by an XML file of approximately 8MB and contains thousands of such links. I am finding that the JVM memory pool required to process this is about 750MB at this point. My database is not yet complete and the documents are likely to grow both in size and number of links a good bit before I am done.

    I have tried running this on a machine with 512MB of memory and it trashed hopelessly for 12 hours without completing. I have 1 machine with 1GB of RAM that I normally reserve as a server that can do the processing in 1/2 an hour, but I am worried that I will lose the ability to create the document as it continues to grow.

    Do you have any suggestions that would help tame the memory utilization? Will the next release of fop be able to process links with a smaller amount of memory?

    Thank you,
    Ed

RE: Memory utilization

Posted by Matthias Fischer <m....@abc-media.de>.
Hi Ed,

we ran into a similar problem a couple of weeks ago. The most suitable
solution that was offered then by our colleagues in the list was to portion
the transformation process. FOP seems to hold its breath until it dives out
of the waters of transformation at the other end of its page master
sequence.
In our case, it wasn't mass data but an editorial document with lots of
graphics, in reference to which I was advised to use headers (of a certain
level) to start a new page master sequence.

I think, some way or the other you could find a similar solution.

Matthias


Dott. Matthias Fischer
abc.Mediaservice GmbH

Nebelhornstraße 8
86807 Buchloe
Tel. (08241) 9686-38
Fax  (08241) 9686-26
http://www.abc-media.de
e-mail: m.fischer@abc-media.de


  -----Original Message-----
  From: Edward Dowgiallo [mailto:edowgial@ptdprolog.net]
  Sent: Thursday, December 06, 2001 7:01 PM
  To: fop-dev@xml.apache.org
  Subject: Memory utilization


  I am attempting to do database publishing using fop to format the output.
I have created a style sheet that creates a table of contents and an index
for my document using <fo:basic-link internal-destination=...> in
combination with id attributes. A larger document is represented by an XML
file of approximately 8MB and contains thousands of such links. I am finding
that the JVM memory pool required to process this is about 750MB at this
point. My database is not yet complete and the documents are likely to grow
both in size and number of links a good bit before I am done.

  I have tried running this on a machine with 512MB of memory and it trashed
hopelessly for 12 hours without completing. I have 1 machine with 1GB of RAM
that I normally reserve as a server that can do the processing in 1/2 an
hour, but I am worried that I will lose the ability to create the document
as it continues to grow.

  Do you have any suggestions that would help tame the memory utilization?
Will the next release of fop be able to process links with a smaller amount
of memory?

  Thank you,
  Ed